Sep 16 17:24:35.467345 ip-10-0-142-164 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 17:24:35.907716 ip-10-0-142-164 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:35.907716 ip-10-0-142-164 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 17:24:35.907716 ip-10-0-142-164 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:35.907716 ip-10-0-142-164 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 17:24:35.907716 ip-10-0-142-164 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:35.909792 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.909696    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.916953    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.916990    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.916994    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.916998    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917001    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917004    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917007    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.916998 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917010    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917013    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917016    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917019    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917021    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917024    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917027    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917031    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917035    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917039    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917042    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917046    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917050    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917053    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917056    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917059    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917062    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917065    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917068    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.917284 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917071    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917074    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917077    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917079    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917089    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917092    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917094    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917097    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917099    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917102    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917104    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917107    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917110    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917112    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917116    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917119    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917121    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917124    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917127    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917129    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.917731 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917132    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917135    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917137    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917140    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917142    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917146    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917149    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917151    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917154    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917156    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917159    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917161    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917164    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917166    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917168    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917171    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917174    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917176    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917179    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917182    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.918259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917184    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917187    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917190    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917192    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917195    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917197    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917200    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917204    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917206    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917209    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917212    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917214    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917217    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917219    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917221    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917224    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917226    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917230    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917233    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.918733 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917235    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917238    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917245    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917247    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917250    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917254    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917256    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917259    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917262    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917265    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917268    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917272    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917275    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.917278    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919076    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919085    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919089    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919092    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919095    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.919205 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919098    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919101    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919105    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919108    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919111    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919113    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919116    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919119    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919122    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919125    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919128    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919131    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919134    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919136    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919139    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919142    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919145    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919148    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919151    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.919668 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919155    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919158    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919161    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919164    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919167    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919170    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919173    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919176    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919179    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919181    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919184    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919186    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919189    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919191    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919194    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919197    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919199    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919202    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919204    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919207    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.920180 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919210    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919213    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919216    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919218    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919221    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919224    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919227    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919230    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919235    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919238    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919240    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919243    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919246    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919249    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919252    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919255    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919258    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919260    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919264    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919266    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.920674 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919269    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919271    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919274    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919277    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919280    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919282    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919285    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919289    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919292    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919295    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919298    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919302    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919305    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919308    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919311    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919314    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919317    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919319    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919322    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.921193 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919325    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919328    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919332    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919336    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919338    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919341    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919344    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919346    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919349    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919352    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919354    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919356    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919359    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919361    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919364    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919367    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919802    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919807    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919810    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.921659 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919813    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919816    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919820    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919823    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919826    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919828    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919831    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919834    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919837    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919839    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919843    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919845    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919848    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919851    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919855    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919857    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919860    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919863    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919865    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919868    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.922144 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919871    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919873    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919876    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919878    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919881    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919883    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919886    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919889    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919891    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919894    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919898    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919901    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919904    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919907    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919910    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919913    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919915    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919918    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919921    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919923    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.922658 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919926    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919929    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919932    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919934    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919939    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919943    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919950    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919953    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919956    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919959    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919962    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919965    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919981    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919984    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919988    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919991    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919994    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.919997    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920000    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.923182 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920003    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920005    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920008    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920011    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920013    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920016    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920018    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920021    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920024    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920026    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920029    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920032    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920034    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920037    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920040    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920042    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920045    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920047    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920050    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920053    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.923644 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920061    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920064    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920066    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920071    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920073    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920076    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920078    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920081    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920083    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920086    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920089    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920091    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920094    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920096    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920099    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920101    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920103    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920178    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920215    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920222    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920227    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 17:24:35.924183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920231    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920235    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920240    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920244    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920262    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920266    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920270    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920273    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920276    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920279    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920282    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920285    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920293    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920297    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920303    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920306    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920309    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920312    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920315    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920318    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920321    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920324    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920328    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920331    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 17:24:35.924703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920334    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920337    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920340    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920343    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920347    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920350    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920353    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920356    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920359    2559 flags.go:64] FLAG: --enable-server="true"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920362    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920368    2559 flags.go:64] FLAG: --event-burst="100"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920371    2559 flags.go:64] FLAG: --event-qps="50"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920374    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920377    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920380    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920388    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920391    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920394    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920397    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920400    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920403    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920407    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920410    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920413    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920416    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 17:24:35.925396 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920419    2559 flags.go:64] FLAG: --feature-gates=""
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920423    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920426    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920429    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920432    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920438    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920441    2559 flags.go:64] FLAG: --help="false"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920444    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-142-164.ec2.internal"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920447    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920450    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920453    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920457    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920460    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920463    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920466    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920469    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920472    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920477    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920480    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920483    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920486    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920489    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920492    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920495    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 17:24:35.926090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920498    2559 flags.go:64] FLAG: --lock-file=""
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920501    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920504    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920507    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920512    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920516    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920519    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920522    2559 flags.go:64] FLAG: --logging-format="text"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920525    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920529    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920531    2559 flags.go:64] FLAG: --manifest-url=""
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920534    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920539    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920544    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920548    2559 flags.go:64] FLAG: --max-pods="110"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920551    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920554    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920557    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920560    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920562    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920566    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920569    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920576    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920579    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920582    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 17:24:35.926684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920586    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920589    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920595    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920597    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920600    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920604    2559 flags.go:64] FLAG: --port="10250"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920607    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920610    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02aa95e0b71aec48c"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920613    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920616    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920619    2559 flags.go:64] FLAG: --register-node="true"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920622    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920626    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920630    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920632    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920635    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920639    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920642    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920645    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920648    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920652    2559 flags.go:64] FLAG: --runonce="false"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920655    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920658    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920661    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920664    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 17:24:35.927336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920667    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920670    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920674    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920677    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920680    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920683    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920686    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920689    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920694    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920697    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920699    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920705    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920707    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920711    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920715    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920718    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920720    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920723    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920726    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920730    2559 flags.go:64] FLAG: --v="2"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920734    2559 flags.go:64] FLAG: --version="false"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920740    2559 flags.go:64] FLAG: --vmodule=""
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920743    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.920747    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920870    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.928230 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920874    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920877    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920881    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920883    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920886    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920889    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920892    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920894    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920897    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920900    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920903    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920905    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920908    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920911    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920913    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920917    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920920    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920923    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920925    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920928    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.929309 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920931    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920934    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920936    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920939    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920942    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920944    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920947    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920950    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920954    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920958    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920961    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920964    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920966    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920982    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920986    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920989    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920992    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920994    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.920997    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921000    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.930252 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921003    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921005    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921008    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921011    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921014    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921016    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921019    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921023    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921026    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921029    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921032    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921035    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921037    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921040    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921043    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921045    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921048    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921051    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921054    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921056    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.930908 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921059    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921062    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921070    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921073    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921075    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921078    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921082    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921085    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921087    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921090    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921093    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921095    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921098    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921101    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921104    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921106    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921109    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921112    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921114    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921119    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.931488 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921121    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921124    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921128    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921132    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921135    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921138    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921141    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921144    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921147    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921150    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921153    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921156    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921159    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921161    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921164    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921167    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921170    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.932323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.921173    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.921180    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.930567    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.930592    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.931567    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.931583    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931696    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931705    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931710    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931715    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931720    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931725    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931731    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931735    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931740    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.933078 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931744    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931748    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931753    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931757    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931761    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931767    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931774    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931779    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931783    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931788    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931792    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931797    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931801    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931805    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931813    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931821    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931826    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931832    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931837    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.933541 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931841    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931846    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931850    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931854    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931859    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931863    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931867    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931871    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931876    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931880    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931884    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931888    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931893    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931897    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931901    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931906    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931910    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931915    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931919    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931923    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.934048 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931927    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931932    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931936    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931940    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931945    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931949    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931954    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931958    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931963    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931984    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931989    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931993    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.931997    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932003    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932007    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932013    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932019    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932025    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932030    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.934781 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932035    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932040    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932044    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932049    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932053    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932058    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932062    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932066    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932071    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932075    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932079    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932084    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932088    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932092    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932097    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932102    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932106    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932110    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932115    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.935410 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932119    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932123    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932127    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932131    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932137    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932142    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932146    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932151    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932155    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932159    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932163    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932167    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932171    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.932180    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932405    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:35.936192 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932414    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932419    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932423    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932427    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932432    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932436    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932440    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932447    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932453    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932459    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932463    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932468    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932472    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932476    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932481    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932485    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932490    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932494    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932499    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:35.936709 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932504    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932508    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932512    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932517    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932521    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932526    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932531    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932535    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932539    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932543    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932550    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932555    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932560    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932565    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932570    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932574    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932578    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932583    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932587    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:35.937248 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932593    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932599    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932604    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932608    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932613    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932617    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932622    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932626    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932630    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932634    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932639    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932643    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932647    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932651    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932656    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932660    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932664    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932669    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932673    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:35.937805 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932678    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932682    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932686    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932690    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932695    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932699    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932703    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932707    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932711    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932715    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932719    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932723    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932727    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932732    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932736    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932740    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932745    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932749    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932753    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932757    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:35.938306 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932761    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932765    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932769    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932773    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932777    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932782    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932786    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932790    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932795    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932799    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932803    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932808    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932812    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932816    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932820    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932824    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932828    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932832    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932837    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:35.938860 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932841    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:35.932845    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.932852    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.933088    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.937233    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.938371    2559 server.go:1019] "Starting client certificate rotation"
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.938476    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 17:24:35.939504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.938509    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 17:24:35.941502 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.941482    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 17:24:35.943021 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.942997    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 17:24:35.949465 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.949447    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 16 17:24:35.954414 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.954393    2559 log.go:25] "Validated CRI v1 image API"
Sep 16 17:24:35.955229 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.955215    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 17:24:35.958508 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.958487    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9045c4ff-d83d-494e-987a-9eefd195d594:/dev/nvme0n1p4 9cdc3695-1b1b-46f3-ba58-a459ea1d32e6:/dev/nvme0n1p3]
Sep 16 17:24:35.958569 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.958507    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 17:24:35.963601 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.963477    2559 manager.go:216] Machine: {Timestamp:2026-09-16 17:24:35.962949636 +0000 UTC m=+0.372318929 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088835 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a117db2f8c3ee647d894d7ba6199 SystemUUID:ec24a117-db2f-8c3e-e647-d894d7ba6199 BootID:5111d814-5d0b-4fcd-8c20-bec6f7f05c97 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cc:eb:32:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cc:eb:32:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:94:7e:04:83:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 17:24:35.963601 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.963594    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 17:24:35.963756 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.963678    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 17:24:35.967528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.967494    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 17:24:35.967673 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.967531    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-164.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 17:24:35.968391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.968379    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 17:24:35.968391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.968392    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 17:24:35.968482 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.968400    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 17:24:35.968482 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.968416    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 17:24:35.969830 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.969815    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 17:24:35.975182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975168    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 17:24:35.975311 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975300    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 17:24:35.975362 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975337    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 17:24:35.975362 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975347    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 17:24:35.975445 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975364    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 17:24:35.975445 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975374    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 16 17:24:35.975445 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975385    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 17:24:35.975756 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975744    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 17:24:35.975799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.975765    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 17:24:35.978493 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.978474    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 17:24:35.979210 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.979193    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 17:24:35.981626 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.981613    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 17:24:35.981684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.981643    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 17:24:35.983732 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983715    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 17:24:35.983788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983744    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 17:24:35.983788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983753    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 17:24:35.983788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983762    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 17:24:35.983788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983775    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 17:24:35.983788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983786    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983805    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983813    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983823    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983831    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983843    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 17:24:35.983992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.983851    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 17:24:35.985718 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.985703    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 17:24:35.985718 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.985718    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 17:24:35.989646 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.989632    2559 server.go:1268] "Started kubelet"
Sep 16 17:24:35.990147 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.990036    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 17:24:35.990621 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.990544    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 17:24:35.990705 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.990646    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 17:24:35.990881 ip-10-0-142-164 systemd[1]: Started Kubernetes Kubelet.
Sep 16 17:24:35.992162 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.992143    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 17:24:35.994485 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.994466    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 17:24:35.995002 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:35.994962    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 17:24:35.997827 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.997810    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 17:24:35.997955 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:35.997940    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 17:24:35.998165 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:35.998148    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 17:24:35.999169 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:35.997959    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d43b090bc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:35.989598396 +0000 UTC m=+0.398967690,LastTimestamp:2026-09-16 17:24:35.989598396 +0000 UTC m=+0.398967690,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:35.999741 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:35.999722    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 17:24:36.000383 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.000368    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 17:24:36.000509 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.000489    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.001285 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.001164    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 17:24:36.001509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.001366    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 17:24:36.001509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.001384    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 17:24:36.001509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.001511    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 17:24:36.001666 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.001518    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 17:24:36.002626 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.002607    2559 factory.go:55] Registering systemd factory
Sep 16 17:24:36.002710 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.002631    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 16 17:24:36.002919 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.002906    2559 factory.go:153] Registering CRI-O factory
Sep 16 17:24:36.002986 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.002923    2559 factory.go:223] Registration of the crio container factory successfully
Sep 16 17:24:36.003039 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.002965    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 17:24:36.003039 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.003009    2559 factory.go:103] Registering Raw factory
Sep 16 17:24:36.003039 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.003011    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 17:24:36.003166 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.003043    2559 manager.go:1196] Started watching for new ooms in manager
Sep 16 17:24:36.003166 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.003141    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 17:24:36.004241 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.004226    2559 manager.go:349] Starting recovery of all containers
Sep 16 17:24:36.016344 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.016159    2559 manager.go:354] Recovery completed
Sep 16 17:24:36.021391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.021374    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.023351 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023333    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.023417 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023363    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.023417 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023375    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.023962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023943    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 17:24:36.023962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023962    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 17:24:36.024115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.023953    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.024115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.024005    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 17:24:36.024750 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.024735    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.024804 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.024763    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.024804 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.024773    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.026136 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.026121    2559 policy_none.go:49] "None policy: Start"
Sep 16 17:24:36.026202 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.026142    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 17:24:36.026202 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.026153    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 17:24:36.028051 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.028033    2559 policy_none.go:47] "Start"
Sep 16 17:24:36.034600 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.034525    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d45b39452  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:36.023350354 +0000 UTC m=+0.432719648,LastTimestamp:2026-09-16 17:24:36.023350354 +0000 UTC m=+0.432719648,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:36.044551 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.044472    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d45b3d828  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:36.02336772 +0000 UTC m=+0.432737014,LastTimestamp:2026-09-16 17:24:36.02336772 +0000 UTC m=+0.432737014,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:36.054340 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.054270    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d45b4038c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:36.023378828 +0000 UTC m=+0.432748122,LastTimestamp:2026-09-16 17:24:36.023378828 +0000 UTC m=+0.432748122,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:36.064827 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.064806    2559 manager.go:337] "Starting Device Plugin manager"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.064857    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.064870    2559 server.go:87] "Starting device plugin registration server"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.065319    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.065333    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.065547    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.065622    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.065631    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.066621    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.066962    2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-164.ec2.internal.18d5dd0d45b39452\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d45b39452  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:36.023350354 +0000 UTC m=+0.432719648,LastTimestamp:2026-09-16 17:24:36.024750553 +0000 UTC m=+0.434119850,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.067105    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.067423    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x66d7"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.074524    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x66d7"
Sep 16 17:24:36.084837 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.083148    2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-164.ec2.internal.18d5dd0d45b3d828\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-164.ec2.internal.18d5dd0d45b3d828  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-164.ec2.internal,UID:ip-10-0-142-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-164.ec2.internal,},FirstTimestamp:2026-09-16 17:24:36.02336772 +0000 UTC m=+0.432737014,LastTimestamp:2026-09-16 17:24:36.0247683 +0000 UTC m=+0.434137593,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-164.ec2.internal,}"
Sep 16 17:24:36.108122 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.108087    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 17:24:36.109316 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.109291    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 17:24:36.109457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.109357    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 17:24:36.109457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.109387    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 17:24:36.109557 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.109493    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 17:24:36.112890 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.112866    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:36.166321 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.166217    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.167353 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.167332    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.167448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.167366    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.167448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.167377    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.167448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.167400    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.175456 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.175436    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.175514 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.175465    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-164.ec2.internal\": node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.201879 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.201854    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.210176 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.210152    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"]
Sep 16 17:24:36.210247 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.210226    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.211212 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.211195    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.211270 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.211228    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.211270 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.211241    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.213607 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.213594    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.213766 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.213744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.213805 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.213784    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.214375 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214352    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.214375 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214378    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.214546 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214396    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.214546 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214413    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.214546 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214440    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.214546 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.214450    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.216869 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.216851    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.216985 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.216886    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:36.217663 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.217644    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:36.217737 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.217674    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:36.217737 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.217684    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:36.239842 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.239815    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-164.ec2.internal\" not found" node="ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.244209 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.244194    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-164.ec2.internal\" not found" node="ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.302136 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.302104    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.403115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:36.403083    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:36.403256 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.403157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.403256 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.403186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:36.403256 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:36.403204    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/174cc1912109f9f57a175c779d7e35e4-config\") pod \"kube-apiserver-proxy-ip-10-0-142-164.ec2.internal\" (UID: \"174cc1912109f9f57a175c779d7e35e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.682866 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:37.682844    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:37.682949 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.682898    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.682949 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.682921    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.682949 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.682938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/174cc1912109f9f57a175c779d7e35e4-config\") pod \"kube-apiserver-proxy-ip-10-0-142-164.ec2.internal\" (UID: \"174cc1912109f9f57a175c779d7e35e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.683082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.683012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/174cc1912109f9f57a175c779d7e35e4-config\") pod \"kube-apiserver-proxy-ip-10-0-142-164.ec2.internal\" (UID: \"174cc1912109f9f57a175c779d7e35e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.683082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.683016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.683082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.683016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f140e30d3c9f86b28b0a48e71913b46c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal\" (UID: \"f140e30d3c9f86b28b0a48e71913b46c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.722099 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.722061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.727298 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:37.727240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:37.783664 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:37.783627    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:37.884246 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:37.884212    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:37.984761 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:37.984666    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:38.085204 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:38.085154    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:38.118368 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.118341    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 17:24:38.118961 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.118589    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 17:24:38.185757 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:38.185723    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:38.261061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.260988    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 17:19:37 +0000 UTC" deadline="2028-05-23 20:47:39.939470306 +0000 UTC"
Sep 16 17:24:38.261061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.261020    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14763h23m1.678454062s"
Sep 16 17:24:38.286325 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:38.286295    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:38.315610 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:38.315576    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf140e30d3c9f86b28b0a48e71913b46c.slice/crio-d36a636b06f6c97e9016d7de5c9e167b0e6e951cb79e4ec7509fd4605fdd96c3 WatchSource:0}: Error finding container d36a636b06f6c97e9016d7de5c9e167b0e6e951cb79e4ec7509fd4605fdd96c3: Status 404 returned error can't find the container with id d36a636b06f6c97e9016d7de5c9e167b0e6e951cb79e4ec7509fd4605fdd96c3
Sep 16 17:24:38.315834 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:38.315817    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174cc1912109f9f57a175c779d7e35e4.slice/crio-58c3ecd2333891778607f69e2054108e32f27e2c1f2ee86c27078b7f80628aed WatchSource:0}: Error finding container 58c3ecd2333891778607f69e2054108e32f27e2c1f2ee86c27078b7f80628aed: Status 404 returned error can't find the container with id 58c3ecd2333891778607f69e2054108e32f27e2c1f2ee86c27078b7f80628aed
Sep 16 17:24:38.321352 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.321336    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:24:38.348031 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.348001    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:38.348916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.348898    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:38.349007 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.348927    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:38.349007 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.348938    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:38.386522 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:38.386492    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-164.ec2.internal\" not found"
Sep 16 17:24:38.400265 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.400243    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:38.480448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.480410    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:38.493066 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.493040    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 17:24:38.495017 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.495001    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal"
Sep 16 17:24:38.511177 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.511127    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 17:24:38.637387 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.637358    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:38.655000 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.654965    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:38.955008 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:38.954972    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:39.156121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.156086    2559 apiserver.go:52] "Watching apiserver"
Sep 16 17:24:39.164339 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.164309    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 17:24:39.164726 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.164695    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-xjdkp","openshift-network-diagnostics/network-check-target-m7czc","openshift-ovn-kubernetes/ovnkube-node-w85dt","kube-system/global-pull-secret-syncer-7c82q","openshift-dns/node-resolver-ndng9","openshift-image-registry/node-ca-hqchc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal","openshift-multus/multus-additional-cni-plugins-6pdp6","openshift-multus/multus-snndz","openshift-multus/network-metrics-daemon-5vgn8","openshift-network-operator/iptables-alerter-dpp2m","kube-system/konnectivity-agent-cfvdg","kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"]
Sep 16 17:24:39.167302 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.167279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.167415 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.167373    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:39.169489 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.169461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.172106 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.172075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.173193 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.173164    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wwr9h\""
Sep 16 17:24:39.173411 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.173371    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.173504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.173448    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.173580 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.173371    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 17:24:39.174512 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.174495    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 17:24:39.174639 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.174616    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.174727 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.174705    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:39.177261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177197    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.177261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177240    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 17:24:39.177406 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177359    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bvgbv\""
Sep 16 17:24:39.177406 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177383    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.177499 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177426    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 17:24:39.177585 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177548    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.177784 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.177717    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 17:24:39.179504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.179478    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.179907 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.179697    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 17:24:39.181814 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.181764    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.184295 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.184275    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.186662 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.186616    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:39.186744 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.186698    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:39.189183 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.189166    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191129    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191246    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191300    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191247    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191353    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191306    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.191475 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191251    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-host-slash\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggkp\" (UniqueName: \"kubernetes.io/projected/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-kube-api-access-6ggkp\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191658    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191703    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-env-overrides\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.191795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-system-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-cnibin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-var-lib-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-netns\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-kubelet\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-hostroot\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.191985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-netd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-script-lib\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192029    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-multus\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-multus-daemon-config\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192108 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6kb\" (UniqueName: \"kubernetes.io/projected/d2f284d7-d259-445b-928e-39708d5bb6ef-kube-api-access-xl6kb\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-dbus\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b59791de-9fae-4f5f-8a78-5cb47514ee7e-host\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192239    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brxwg\" (UniqueName: \"kubernetes.io/projected/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-kube-api-access-brxwg\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192272    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dz7gd\""
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-socket-dir-parent\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-conf-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192363    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-systemd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-node-log\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-bin\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-multus-certs\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192478    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-iptables-alerter-script\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192510    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192517    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n7cgl\""
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1909d2d2-d9ad-4568-a352-55f882e0f6cb-tmp-dir\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.192740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192578    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvc2r\" (UniqueName: \"kubernetes.io/projected/a453bf53-5455-484f-a121-7037b4f9eafa-kube-api-access-gvc2r\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192684    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-kubelet-config\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192717    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-749hx\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b59791de-9fae-4f5f-8a78-5cb47514ee7e-serviceca\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192771    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lnjxd\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192768    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzmnz\" (UniqueName: \"kubernetes.io/projected/b59791de-9fae-4f5f-8a78-5cb47514ee7e-kube-api-access-mzmnz\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-systemd-units\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192838    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jhh58\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192837    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-netns\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-etc-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192901    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192904    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-ovn\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-log-socket\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-cni-binary-copy\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.192980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-kubelet\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-slash\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.193736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193046    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-config\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74jrx\" (UniqueName: \"kubernetes.io/projected/1909d2d2-d9ad-4568-a352-55f882e0f6cb-kube-api-access-74jrx\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-k8s-cni-cncf-io\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-bin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-etc-kubernetes\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovn-node-metrics-cert\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1909d2d2-d9ad-4568-a352-55f882e0f6cb-hosts-file\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.194661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.193259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.195760 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.195741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.196907 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.196709    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 17:24:39.196907 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.196760    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 17:24:39.196907 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.196787    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8ksm4\""
Sep 16 17:24:39.198874 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.198854    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 17:24:39.199399 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.199253    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mwpng\""
Sep 16 17:24:39.199399 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.199365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 17:24:39.218985 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.218958    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 17:24:39.249096 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.249063    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgtfw"
Sep 16 17:24:39.272695 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.272659    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgtfw"
Sep 16 17:24:39.282393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.282364    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 17:24:39.293432 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-iptables-alerter-script\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.293432 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1909d2d2-d9ad-4568-a352-55f882e0f6cb-tmp-dir\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293586    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysconfig\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-cni-binary-copy\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.293697 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgfk8\" (UniqueName: \"kubernetes.io/projected/3323f05e-e29b-4b3d-9142-e08ae2a08824-kube-api-access-bgfk8\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-modprobe-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bcc847-b121-40de-86f0-bc6dbafa1731-konnectivity-ca\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-kubelet\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-slash\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-kubelet\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293865    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1909d2d2-d9ad-4568-a352-55f882e0f6cb-tmp-dir\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74jrx\" (UniqueName: \"kubernetes.io/projected/1909d2d2-d9ad-4568-a352-55f882e0f6cb-kube-api-access-74jrx\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293907    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-slash\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293908    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-etc-selinux\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovn-node-metrics-cert\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.293996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.293985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-host-slash\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggkp\" (UniqueName: \"kubernetes.io/projected/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-kube-api-access-6ggkp\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294039    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294058    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-iptables-alerter-script\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294076    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-host-slash\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-env-overrides\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-cnibin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-kubelet\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-socket-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-netns\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-sys\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-netd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-multus-daemon-config\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-cnibin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.294523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-kubelet\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-cni-binary-copy\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294365    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6kb\" (UniqueName: \"kubernetes.io/projected/d2f284d7-d259-445b-928e-39708d5bb6ef-kube-api-access-xl6kb\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294418    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.294441    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmc5q\" (UniqueName: \"kubernetes.io/projected/b70ed42a-ce9e-48b4-a82e-eac803af9cda-kube-api-access-gmc5q\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294535    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-netns\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.294541    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:24:39.794502517 +0000 UTC m=+3.024284202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-systemd\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294686    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal" event={"ID":"f140e30d3c9f86b28b0a48e71913b46c","Type":"ContainerStarted","Data":"d36a636b06f6c97e9016d7de5c9e167b0e6e951cb79e4ec7509fd4605fdd96c3"}
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-lib-modules\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-socket-dir-parent\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-env-overrides\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a453bf53-5455-484f-a121-7037b4f9eafa-multus-daemon-config\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-netd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-system-cni-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.295252 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-socket-dir-parent\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294885    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-var-lib-kubelet\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-systemd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-multus-certs\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-systemd\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.294961    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-multus-certs\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-device-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.295031    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.295084    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:24:39.795069838 +0000 UTC m=+3.024851508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvc2r\" (UniqueName: \"kubernetes.io/projected/a453bf53-5455-484f-a121-7037b4f9eafa-kube-api-access-gvc2r\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-kubelet-config\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b59791de-9fae-4f5f-8a78-5cb47514ee7e-serviceca\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-systemd-units\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295176    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-netns\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-kubelet-config\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.296115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-etc-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-ovn\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-log-socket\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-registration-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295283    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-netns\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cnibin\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-host\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-log-socket\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-etc-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-ovn\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzmnz\" (UniqueName: \"kubernetes.io/projected/b59791de-9fae-4f5f-8a78-5cb47514ee7e-kube-api-access-mzmnz\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295407    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-config\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295429    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-k8s-cni-cncf-io\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-bin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-run-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-etc-kubernetes\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.296908 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-etc-kubernetes\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295493    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-dbus\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295515    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b59791de-9fae-4f5f-8a78-5cb47514ee7e-host\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295533    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1909d2d2-d9ad-4568-a352-55f882e0f6cb-hosts-file\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b59791de-9fae-4f5f-8a78-5cb47514ee7e-serviceca\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295552    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-system-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-hostroot\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-kubernetes\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-tmp\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bcc847-b121-40de-86f0-bc6dbafa1731-agent-certs\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-system-cni-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-var-lib-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-sys-fs\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b59791de-9fae-4f5f-8a78-5cb47514ee7e-host\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295712    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kffwl\" (UniqueName: \"kubernetes.io/projected/364d30b1-1532-4731-8502-a58958b805f3-kube-api-access-kffwl\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1909d2d2-d9ad-4568-a352-55f882e0f6cb-hosts-file\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-script-lib\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.297611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-multus\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cea32ac-e0c2-4c23-8873-acdedd71424b-dbus\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295781    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-conf\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-run\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-config\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-multus\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-var-lib-openvswitch\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-hostroot\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-run-k8s-cni-cncf-io\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brxwg\" (UniqueName: \"kubernetes.io/projected/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-kube-api-access-brxwg\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-host-var-lib-cni-bin\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.295914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-conf-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovnkube-script-lib\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-etc-tuned\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a453bf53-5455-484f-a121-7037b4f9eafa-multus-conf-dir\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-node-log\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.298393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-node-log\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-systemd-units\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-bin\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-host-cni-bin\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296398    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.296883    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal" event={"ID":"174cc1912109f9f57a175c779d7e35e4","Type":"ContainerStarted","Data":"58c3ecd2333891778607f69e2054108e32f27e2c1f2ee86c27078b7f80628aed"}
Sep 16 17:24:39.299263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.298670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-ovn-node-metrics-cert\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.303466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.303442    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74jrx\" (UniqueName: \"kubernetes.io/projected/1909d2d2-d9ad-4568-a352-55f882e0f6cb-kube-api-access-74jrx\") pod \"node-resolver-ndng9\" (UID: \"1909d2d2-d9ad-4568-a352-55f882e0f6cb\") " pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.306297 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.306260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvc2r\" (UniqueName: \"kubernetes.io/projected/a453bf53-5455-484f-a121-7037b4f9eafa-kube-api-access-gvc2r\") pod \"multus-snndz\" (UID: \"a453bf53-5455-484f-a121-7037b4f9eafa\") " pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.306432 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.306412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggkp\" (UniqueName: \"kubernetes.io/projected/1a5c8ef7-47d1-4061-9a1f-33169729a3d0-kube-api-access-6ggkp\") pod \"iptables-alerter-dpp2m\" (UID: \"1a5c8ef7-47d1-4061-9a1f-33169729a3d0\") " pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.308263 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.308242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brxwg\" (UniqueName: \"kubernetes.io/projected/558c4e98-bb49-4d04-a4e9-6d6b94a787b7-kube-api-access-brxwg\") pod \"ovnkube-node-w85dt\" (UID: \"558c4e98-bb49-4d04-a4e9-6d6b94a787b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.312316 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.312297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzmnz\" (UniqueName: \"kubernetes.io/projected/b59791de-9fae-4f5f-8a78-5cb47514ee7e-kube-api-access-mzmnz\") pod \"node-ca-hqchc\" (UID: \"b59791de-9fae-4f5f-8a78-5cb47514ee7e\") " pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.314784 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.314759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6kb\" (UniqueName: \"kubernetes.io/projected/d2f284d7-d259-445b-928e-39708d5bb6ef-kube-api-access-xl6kb\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.397675 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.397675 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysconfig\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgfk8\" (UniqueName: \"kubernetes.io/projected/3323f05e-e29b-4b3d-9142-e08ae2a08824-kube-api-access-bgfk8\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-modprobe-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bcc847-b121-40de-86f0-bc6dbafa1731-konnectivity-ca\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-etc-selinux\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-socket-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysconfig\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.397916 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-sys\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397939    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmc5q\" (UniqueName: \"kubernetes.io/projected/b70ed42a-ce9e-48b4-a82e-eac803af9cda-kube-api-access-gmc5q\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397965    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-systemd\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.397989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-lib-modules\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-system-cni-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-var-lib-kubelet\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-device-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-registration-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398225    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cnibin\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398271    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-host\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-sys\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-kubernetes\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398331    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-tmp\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bcc847-b121-40de-86f0-bc6dbafa1731-agent-certs\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.398450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-sys-fs\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398419    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kffwl\" (UniqueName: \"kubernetes.io/projected/364d30b1-1532-4731-8502-a58958b805f3-kube-api-access-kffwl\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398457    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-conf\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398446    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-socket-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-registration-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-run\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-system-cni-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-run\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398621    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-var-lib-kubelet\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bcc847-b121-40de-86f0-bc6dbafa1731-konnectivity-ca\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cnibin\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-device-dir\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398359    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-etc-selinux\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398301    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-lib-modules\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-systemd\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.398839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-etc-tuned\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3323f05e-e29b-4b3d-9142-e08ae2a08824-sys-fs\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b70ed42a-ce9e-48b4-a82e-eac803af9cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399228    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b70ed42a-ce9e-48b4-a82e-eac803af9cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-sysctl-conf\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-kubernetes\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399392    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-host\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.399956 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.399477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/364d30b1-1532-4731-8502-a58958b805f3-etc-modprobe-d\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.401392 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.401362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-tmp\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.401520 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.401436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/364d30b1-1532-4731-8502-a58958b805f3-etc-tuned\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.401520 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.401502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bcc847-b121-40de-86f0-bc6dbafa1731-agent-certs\") pod \"konnectivity-agent-cfvdg\" (UID: \"05bcc847-b121-40de-86f0-bc6dbafa1731\") " pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.404652 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.404630    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:39.404760 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.404658    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:39.404760 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.404672    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:39.404760 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.404738    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:39.904719494 +0000 UTC m=+3.134501185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:39.413976 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.413937    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmc5q\" (UniqueName: \"kubernetes.io/projected/b70ed42a-ce9e-48b4-a82e-eac803af9cda-kube-api-access-gmc5q\") pod \"multus-additional-cni-plugins-6pdp6\" (UID: \"b70ed42a-ce9e-48b4-a82e-eac803af9cda\") " pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.414445 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.414422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kffwl\" (UniqueName: \"kubernetes.io/projected/364d30b1-1532-4731-8502-a58958b805f3-kube-api-access-kffwl\") pod \"tuned-xjdkp\" (UID: \"364d30b1-1532-4731-8502-a58958b805f3\") " pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.415095 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.415070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgfk8\" (UniqueName: \"kubernetes.io/projected/3323f05e-e29b-4b3d-9142-e08ae2a08824-kube-api-access-bgfk8\") pod \"aws-ebs-csi-driver-node-z5vr9\" (UID: \"3323f05e-e29b-4b3d-9142-e08ae2a08824\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.485005 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.484920    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dpp2m"
Sep 16 17:24:39.491657 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.491626    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:24:39.502009 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.501968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndng9"
Sep 16 17:24:39.507092 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.507071    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hqchc"
Sep 16 17:24:39.515063 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.515034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-snndz"
Sep 16 17:24:39.522928 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.522904    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9"
Sep 16 17:24:39.529852 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.529832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xjdkp"
Sep 16 17:24:39.536763 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.536740    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6pdp6"
Sep 16 17:24:39.542744 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.542724    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:39.802272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.802190    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:39.802272 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:39.802229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:39.802442 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.802333    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:39.802442 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.802335    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:39.802442 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.802387    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:24:40.802374325 +0000 UTC m=+4.032155999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:39.802442 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:39.802401    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:24:40.802394618 +0000 UTC m=+4.032176289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:40.003262 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.003223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:40.003449 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.003428    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:40.003498 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.003461    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:40.003498 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.003475    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:40.003604 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.003542    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:41.003523798 +0000 UTC m=+4.233305482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:40.152323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.152298    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70ed42a_ce9e_48b4_a82e_eac803af9cda.slice/crio-fce188bab6d3fa58fa1d240cbba083a75a330ea5d9e7f90cb1b3484a30836cde WatchSource:0}: Error finding container fce188bab6d3fa58fa1d240cbba083a75a330ea5d9e7f90cb1b3484a30836cde: Status 404 returned error can't find the container with id fce188bab6d3fa58fa1d240cbba083a75a330ea5d9e7f90cb1b3484a30836cde
Sep 16 17:24:40.152941 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.152923    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bcc847_b121_40de_86f0_bc6dbafa1731.slice/crio-21b1ab17ffc752fa06bd2ab3d1a8e6aa4eb96f5d673da919ed8bc1e20d29c435 WatchSource:0}: Error finding container 21b1ab17ffc752fa06bd2ab3d1a8e6aa4eb96f5d673da919ed8bc1e20d29c435: Status 404 returned error can't find the container with id 21b1ab17ffc752fa06bd2ab3d1a8e6aa4eb96f5d673da919ed8bc1e20d29c435
Sep 16 17:24:40.154705 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.154683    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda453bf53_5455_484f_a121_7037b4f9eafa.slice/crio-80a3474c9d8e200bdf7588bfacc94f503e92ea116a232b7f5cbe251307abc4eb WatchSource:0}: Error finding container 80a3474c9d8e200bdf7588bfacc94f503e92ea116a232b7f5cbe251307abc4eb: Status 404 returned error can't find the container with id 80a3474c9d8e200bdf7588bfacc94f503e92ea116a232b7f5cbe251307abc4eb
Sep 16 17:24:40.158609 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.158584    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5c8ef7_47d1_4061_9a1f_33169729a3d0.slice/crio-9ed025bba4b52b29e93425123823afc410aa899920e45aed6792fb6e1451cfbc WatchSource:0}: Error finding container 9ed025bba4b52b29e93425123823afc410aa899920e45aed6792fb6e1451cfbc: Status 404 returned error can't find the container with id 9ed025bba4b52b29e93425123823afc410aa899920e45aed6792fb6e1451cfbc
Sep 16 17:24:40.159581 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.159536    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3323f05e_e29b_4b3d_9142_e08ae2a08824.slice/crio-06d397268f5544b71cac59e783bca6f1b8a33ebb092bb48e043858f1b632503c WatchSource:0}: Error finding container 06d397268f5544b71cac59e783bca6f1b8a33ebb092bb48e043858f1b632503c: Status 404 returned error can't find the container with id 06d397268f5544b71cac59e783bca6f1b8a33ebb092bb48e043858f1b632503c
Sep 16 17:24:40.161046 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.160453    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59791de_9fae_4f5f_8a78_5cb47514ee7e.slice/crio-a59f302cf9245e3b13720e8a99c673e67b1492757e45717e09a276be0f38f779 WatchSource:0}: Error finding container a59f302cf9245e3b13720e8a99c673e67b1492757e45717e09a276be0f38f779: Status 404 returned error can't find the container with id a59f302cf9245e3b13720e8a99c673e67b1492757e45717e09a276be0f38f779
Sep 16 17:24:40.161501 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.161483    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558c4e98_bb49_4d04_a4e9_6d6b94a787b7.slice/crio-7488b29d7be0f88291078f31d032f4f241d9642e9d111ecedf606a5a0d39c000 WatchSource:0}: Error finding container 7488b29d7be0f88291078f31d032f4f241d9642e9d111ecedf606a5a0d39c000: Status 404 returned error can't find the container with id 7488b29d7be0f88291078f31d032f4f241d9642e9d111ecedf606a5a0d39c000
Sep 16 17:24:40.163259 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:24:40.163240    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1909d2d2_d9ad_4568_a352_55f882e0f6cb.slice/crio-ce40f46a95b8fc2cbeae57d0488f17592b2db9feb9644b629fe34372633ba0a9 WatchSource:0}: Error finding container ce40f46a95b8fc2cbeae57d0488f17592b2db9feb9644b629fe34372633ba0a9: Status 404 returned error can't find the container with id ce40f46a95b8fc2cbeae57d0488f17592b2db9feb9644b629fe34372633ba0a9
Sep 16 17:24:40.276643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.276437    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 17:19:39 +0000 UTC" deadline="2028-04-19 00:15:23.740568195 +0000 UTC"
Sep 16 17:24:40.276643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.276639    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13926h50m43.463934868s"
Sep 16 17:24:40.300380 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.300345    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dpp2m" event={"ID":"1a5c8ef7-47d1-4061-9a1f-33169729a3d0","Type":"ContainerStarted","Data":"9ed025bba4b52b29e93425123823afc410aa899920e45aed6792fb6e1451cfbc"}
Sep 16 17:24:40.301521 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.301493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xjdkp" event={"ID":"364d30b1-1532-4731-8502-a58958b805f3","Type":"ContainerStarted","Data":"5f3476bc8522911acdb2d8f56d0f4a746492d8331834b0e049d835fd7f57a198"}
Sep 16 17:24:40.302509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.302486    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfvdg" event={"ID":"05bcc847-b121-40de-86f0-bc6dbafa1731","Type":"ContainerStarted","Data":"21b1ab17ffc752fa06bd2ab3d1a8e6aa4eb96f5d673da919ed8bc1e20d29c435"}
Sep 16 17:24:40.303604 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.303583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerStarted","Data":"fce188bab6d3fa58fa1d240cbba083a75a330ea5d9e7f90cb1b3484a30836cde"}
Sep 16 17:24:40.305893 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.305846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal" event={"ID":"174cc1912109f9f57a175c779d7e35e4","Type":"ContainerStarted","Data":"c2cabbd54e4d915dcab4084c47e8c67341f75eae0950db116b8d9393db9a30cf"}
Sep 16 17:24:40.307049 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.307025    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndng9" event={"ID":"1909d2d2-d9ad-4568-a352-55f882e0f6cb","Type":"ContainerStarted","Data":"ce40f46a95b8fc2cbeae57d0488f17592b2db9feb9644b629fe34372633ba0a9"}
Sep 16 17:24:40.308376 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.308354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"7488b29d7be0f88291078f31d032f4f241d9642e9d111ecedf606a5a0d39c000"}
Sep 16 17:24:40.309522 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.309493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9" event={"ID":"3323f05e-e29b-4b3d-9142-e08ae2a08824","Type":"ContainerStarted","Data":"06d397268f5544b71cac59e783bca6f1b8a33ebb092bb48e043858f1b632503c"}
Sep 16 17:24:40.310619 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.310597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-snndz" event={"ID":"a453bf53-5455-484f-a121-7037b4f9eafa","Type":"ContainerStarted","Data":"80a3474c9d8e200bdf7588bfacc94f503e92ea116a232b7f5cbe251307abc4eb"}
Sep 16 17:24:40.311720 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.311697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hqchc" event={"ID":"b59791de-9fae-4f5f-8a78-5cb47514ee7e","Type":"ContainerStarted","Data":"a59f302cf9245e3b13720e8a99c673e67b1492757e45717e09a276be0f38f779"}
Sep 16 17:24:40.324014 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.323964    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-164.ec2.internal" podStartSLOduration=2.323489167 podStartE2EDuration="2.323489167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:40.323329193 +0000 UTC m=+3.553110884" watchObservedRunningTime="2026-09-16 17:24:40.323489167 +0000 UTC m=+3.553270858"
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.810363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:40.810427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.810541    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.810619    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:24:42.810601302 +0000 UTC m=+6.040382978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.810993    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:40.811115 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:40.811034    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:24:42.811020776 +0000 UTC m=+6.040802449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:41.015842 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.015803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:41.016102 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.016015    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:41.016102 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.016041    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:41.016102 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.016056    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:41.016265 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.016120    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:43.016100598 +0000 UTC m=+6.245882272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:41.277849 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.277769    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 17:19:39 +0000 UTC" deadline="2028-04-07 00:30:52.812389917 +0000 UTC"
Sep 16 17:24:41.277849 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.277817    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13639h6m11.534580041s"
Sep 16 17:24:41.289819 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.289310    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:41.289819 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.289328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:41.289819 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.289450    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:41.289819 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.289551    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:41.290392 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.290236    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:41.290392 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:41.290350    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:41.336790 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.336746    2559 generic.go:356] "Generic (PLEG): container finished" podID="f140e30d3c9f86b28b0a48e71913b46c" containerID="1cf7992730ccc5a7f8f0dde2e30a8c55f17bcc1354b70b478905ecadb07dbab5" exitCode=0
Sep 16 17:24:41.337763 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:41.337736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal" event={"ID":"f140e30d3c9f86b28b0a48e71913b46c","Type":"ContainerDied","Data":"1cf7992730ccc5a7f8f0dde2e30a8c55f17bcc1354b70b478905ecadb07dbab5"}
Sep 16 17:24:42.364853 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:42.364718    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal" event={"ID":"f140e30d3c9f86b28b0a48e71913b46c","Type":"ContainerStarted","Data":"cab7346f408bfa409f779a430e7a0774ea9a280206092d1b8b701c212604b233"}
Sep 16 17:24:42.381491 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:42.381432    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-164.ec2.internal" podStartSLOduration=4.381409541 podStartE2EDuration="4.381409541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:42.381014675 +0000 UTC m=+5.610796366" watchObservedRunningTime="2026-09-16 17:24:42.381409541 +0000 UTC m=+5.611191232"
Sep 16 17:24:42.836098 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:42.836053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:42.836282 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:42.836120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:42.836340 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:42.836297    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:42.836389 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:42.836366    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:24:46.836345435 +0000 UTC m=+10.066127103 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:42.836818 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:42.836794    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:42.836978 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:42.836857    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:24:46.836840779 +0000 UTC m=+10.066622456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:43.039232 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:43.039192    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:43.039448 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.039427    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:43.039519 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.039452    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:43.039519 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.039465    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:43.039644 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.039528    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:47.039507857 +0000 UTC m=+10.269289543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:43.289546    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:43.289684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.289685    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:43.289569    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.289762    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:43.290103 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:43.289844    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:45.290065 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:45.290029    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:45.290569 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:45.290188    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:45.290646 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:45.290615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:45.290747 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:45.290712    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:45.290875 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:45.290773    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:45.290953 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:45.290867    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:46.873876 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:46.873837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:46.874324 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:46.873890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:46.874324 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:46.874005    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:46.874324 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:46.874031    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:46.874324 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:46.874081    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:24:54.87405948 +0000 UTC m=+18.103841160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:46.874324 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:46.874101    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:24:54.874091477 +0000 UTC m=+18.103873151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:47.075227 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:47.075166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:47.075392 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.075324    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:47.075392 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.075352    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:47.075392 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.075367    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:47.075539 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.075440    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:55.075419663 +0000 UTC m=+18.305201346 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:47.290920    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:47.291018    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:47.291058    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.291135    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.291492    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:47.291683 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:47.291601    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:49.289468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:49.289429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:49.289468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:49.289465    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:49.289935 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:49.289435    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:49.289935 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:49.289534    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:49.289935 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:49.289666    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:49.289935 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:49.289751    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:51.301868 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:51.301808    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:51.301868 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:51.301815    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:51.302388 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:51.301945    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:51.302388 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:51.301946    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:51.302388 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:51.302065    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:51.302388 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:51.302136    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:53.290031 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:53.290005    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:53.290532 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:53.290002    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:53.290532 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:53.290118    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:53.290532 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:53.290193    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:53.290532 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:53.290008    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:53.290532 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:53.290283    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:54.936641 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:54.936358    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:54.937158 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:54.936673    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:54.937158 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:54.936536    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:54.937158 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:54.936794    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:25:10.936768055 +0000 UTC m=+34.166549731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:54.937158 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:54.936815    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:54.937158 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:54.936875    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:25:10.93686197 +0000 UTC m=+34.166643641 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:55.138807 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:55.138768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:55.138983 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.138931    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:55.138983 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.138957    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:55.138983 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.138969    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:55.139105 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.139039    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:11.139019947 +0000 UTC m=+34.368801619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:55.289466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:55.289376    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:55.289466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:55.289418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:55.289466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:55.289432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:55.289745 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.289514    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:55.289745 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.289581    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:55.289745 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:55.289691    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:57.290321 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:57.290285    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:57.290782 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:57.290391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:57.290782 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:57.290437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:57.290782 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:57.290437    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:57.290782 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:57.290511    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:57.290782 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:57.290625    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:58.400254 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.399989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfvdg" event={"ID":"05bcc847-b121-40de-86f0-bc6dbafa1731","Type":"ContainerStarted","Data":"194cd270bc564fe5bea51117949060abe6de1f5880b039905fb144f9053517e8"}
Sep 16 17:24:58.401818 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.401793    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="5ce9da842512e17f89f60012029df0b1869433239ebc263332053c85e18bb26a" exitCode=0
Sep 16 17:24:58.401932 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.401880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"5ce9da842512e17f89f60012029df0b1869433239ebc263332053c85e18bb26a"}
Sep 16 17:24:58.403448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.403424    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndng9" event={"ID":"1909d2d2-d9ad-4568-a352-55f882e0f6cb","Type":"ContainerStarted","Data":"ef300d5b423254ea7915b2691660b3bc04c228d526bc5901b9a05ada03857e96"}
Sep 16 17:24:58.406201 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.406178    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"210ed6bee66a6edc3f6b22af9b69716963cd4d73bb02d015241cb59f3ae11a79"}
Sep 16 17:24:58.406279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.406253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"5cf602d4e466b69bd72399e9118951db5af48466f25c5e2a35d528d6d61213d9"}
Sep 16 17:24:58.406318 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.406306    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"50b4c1e397fc7abfdc6cb11c734e1185e83e3e68f4d0f910cbb1cbf5329f65ea"}
Sep 16 17:24:58.407867 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.407846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9" event={"ID":"3323f05e-e29b-4b3d-9142-e08ae2a08824","Type":"ContainerStarted","Data":"36ea7a792ad2a1b44c69818766a8b2384be35255ee7c35ec4efa232edcc165af"}
Sep 16 17:24:58.409513 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.409488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-snndz" event={"ID":"a453bf53-5455-484f-a121-7037b4f9eafa","Type":"ContainerStarted","Data":"500102dd84a2a38441783937bba040799629048881e5b7aaa85c04707f244a7e"}
Sep 16 17:24:58.411056 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.411030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hqchc" event={"ID":"b59791de-9fae-4f5f-8a78-5cb47514ee7e","Type":"ContainerStarted","Data":"2d7dc350c0f0cf2fcf3fa08694c45d08ad94269f8fdc2a152f40c8c12e1af2fc"}
Sep 16 17:24:58.412479 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.412459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xjdkp" event={"ID":"364d30b1-1532-4731-8502-a58958b805f3","Type":"ContainerStarted","Data":"691e9fd66e135754536db3f69ab932199addb4c45bf9c6fa67ba86a78bf88624"}
Sep 16 17:24:58.455162 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.455110    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cfvdg" podStartSLOduration=11.150124146 podStartE2EDuration="21.455091737s" totalImagesPullingTime="10.304967591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:58.437489926 +0000 UTC m=+21.667271617" watchObservedRunningTime="2026-09-16 17:24:58.455091737 +0000 UTC m=+21.684873428"
Sep 16 17:24:58.490930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.490879    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xjdkp" podStartSLOduration=3.747652742 podStartE2EDuration="21.490861234s" totalImagesPullingTime="17.743208492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:58.455353503 +0000 UTC m=+21.685135219" watchObservedRunningTime="2026-09-16 17:24:58.490861234 +0000 UTC m=+21.720642925"
Sep 16 17:24:58.563098 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.563053    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ndng9" podStartSLOduration=3.829003675 podStartE2EDuration="21.563036097s" totalImagesPullingTime="17.734032422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:58.51825573 +0000 UTC m=+21.748037432" watchObservedRunningTime="2026-09-16 17:24:58.563036097 +0000 UTC m=+21.792817784"
Sep 16 17:24:58.563207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.563130    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hqchc" podStartSLOduration=3.826192846 podStartE2EDuration="21.563124258s" totalImagesPullingTime="17.736931412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:58.557107975 +0000 UTC m=+21.786889667" watchObservedRunningTime="2026-09-16 17:24:58.563124258 +0000 UTC m=+21.792905950"
Sep 16 17:24:58.628215 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.626048    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-snndz" podStartSLOduration=3.748722704 podStartE2EDuration="21.626028011s" totalImagesPullingTime="17.877305307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:58.625482519 +0000 UTC m=+21.855264212" watchObservedRunningTime="2026-09-16 17:24:58.626028011 +0000 UTC m=+21.855809706"
Sep 16 17:24:58.878647 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.878609    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:58.879626 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:58.879600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:24:59.290226 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.290145    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:24:59.290464 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.290145    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:24:59.290464 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:59.290299    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:24:59.290464 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:59.290356    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:24:59.290464 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.290154    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:24:59.290464 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:24:59.290448    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:24:59.418128 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.418097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"419e3739440a9c4bf8318f5206be765a6c3a069c84428a61489b2f81521a2809"}
Sep 16 17:24:59.418913 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.418168    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"4dc6b7cfb6994e64b241fcfc0bf0cec44f895a829fdc65feb3e08ed66c07a736"}
Sep 16 17:24:59.418913 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.418210    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"7d735d68071b380cd00af23c843fcb470c51acd42d146c6088a9f2fc9a58deaf"}
Sep 16 17:24:59.486372 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:24:59.486340    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 17:25:00.258089 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.257961    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T17:24:59.486364093Z","UUID":"ed5ead41-433a-46e6-a1e9-b570a0888a8d","Handler":null,"Name":"","Endpoint":""}
Sep 16 17:25:00.259785 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.259759    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 17:25:00.259785 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.259794    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 17:25:00.423160 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.423129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9" event={"ID":"3323f05e-e29b-4b3d-9142-e08ae2a08824","Type":"ContainerStarted","Data":"b76abeff16a67592a3761087127cbb3355e198c43d02fc8ed22c9a7c7f8ac7d8"}
Sep 16 17:25:00.424979 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.424955    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 17:25:00.425112 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:00.424990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dpp2m" event={"ID":"1a5c8ef7-47d1-4061-9a1f-33169729a3d0","Type":"ContainerStarted","Data":"1546d1255e448c91054f39589e892a50e89268ec86f0bc7148b66666ff2d997c"}
Sep 16 17:25:01.290065 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.289866    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:01.290232 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.289866    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:01.290232 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:01.290173    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:25:01.290232 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.289902    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:01.290364 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:01.290202    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:25:01.290364 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:01.290323    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:25:01.431216 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.431176    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"a7a546b7ec6bfd1edbc8864ecc7fb04ac6bc6e4b762e5a58a5dce322d5f2b020"}
Sep 16 17:25:01.433593 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.433547    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9" event={"ID":"3323f05e-e29b-4b3d-9142-e08ae2a08824","Type":"ContainerStarted","Data":"a2efad046913096ce6673ac79d2ac6fcb7e59bd5d76628f4292e95bc87a6cd31"}
Sep 16 17:25:01.485960 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:01.485902    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dpp2m" podStartSLOduration=6.747175411 podStartE2EDuration="24.485884502s" totalImagesPullingTime="17.738709091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:00.51425347 +0000 UTC m=+23.744035160" watchObservedRunningTime="2026-09-16 17:25:01.485884502 +0000 UTC m=+24.715666193"
Sep 16 17:25:03.292444 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.292230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:03.292887 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.292230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:03.292887 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:03.292479    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:25:03.292887 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.292240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:03.292887 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:03.292535    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:25:03.292887 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:03.292639    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:25:03.440114 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.440085    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="e4876861c471e7ca386a2328d6c27a7a13183fcf5bc08a992182185568273eae" exitCode=0
Sep 16 17:25:03.440294 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.440172    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"e4876861c471e7ca386a2328d6c27a7a13183fcf5bc08a992182185568273eae"}
Sep 16 17:25:03.447376 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.447354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" event={"ID":"558c4e98-bb49-4d04-a4e9-6d6b94a787b7","Type":"ContainerStarted","Data":"c35140cc967c91e1889bd90b02dacd6e04f00564da0097e851675a162f8c0b3d"}
Sep 16 17:25:03.447604 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.447589    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:03.447676 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.447644    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:03.463425 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.463400    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:03.463650 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.463499    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:03.489129 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.489076    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z5vr9" podStartSLOduration=6.424355926 podStartE2EDuration="26.489059805s" totalImagesPullingTime="20.064703879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:01.485479722 +0000 UTC m=+24.715261422" watchObservedRunningTime="2026-09-16 17:25:03.489059805 +0000 UTC m=+26.718841507"
Sep 16 17:25:03.535884 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:03.535829    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt" podStartSLOduration=8.774241383 podStartE2EDuration="26.535813917s" totalImagesPullingTime="17.761572534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:03.535436759 +0000 UTC m=+26.765218451" watchObservedRunningTime="2026-09-16 17:25:03.535813917 +0000 UTC m=+26.765595608"
Sep 16 17:25:04.363994 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.363903    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:25:04.364585 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.364076    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 17:25:04.364585 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.364500    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cfvdg"
Sep 16 17:25:04.452127 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.452096    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="d8f97be96b9044472f6faa44da39f502eb0e1fe08eb843084b949278d14db90f" exitCode=0
Sep 16 17:25:04.452289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.452181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"d8f97be96b9044472f6faa44da39f502eb0e1fe08eb843084b949278d14db90f"}
Sep 16 17:25:04.452391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:04.452378    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 17:25:05.220473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.220437    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5vgn8"]
Sep 16 17:25:05.220684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.220582    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:05.220731 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:05.220679    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:25:05.237410 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.237373    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7c82q"]
Sep 16 17:25:05.237585 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.237491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:05.237647 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:05.237588    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:25:05.246001 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.245969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7czc"]
Sep 16 17:25:05.246168 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.246101    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:05.246250 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:05.246210    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:25:05.456519 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.456487    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="ad959d10d9f2ea9f5acd0d9fff47451cdb530d30a9348dd84d0a81291ae5b333" exitCode=0
Sep 16 17:25:05.457308 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.456588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"ad959d10d9f2ea9f5acd0d9fff47451cdb530d30a9348dd84d0a81291ae5b333"}
Sep 16 17:25:05.457308 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:05.456690    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 17:25:06.879580 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:06.879532    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:07.290575 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:07.290271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:07.290575 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:07.290352    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:07.290772 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:07.290396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:07.290772 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:07.290633    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:25:07.290772 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:07.290705    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:25:07.290926 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:07.290775    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:25:09.289871 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:09.289838    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:09.290225 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:09.289838    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:09.290225 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:09.289947    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7czc" podUID="be040180-d5b1-45e6-8907-33e1302ae1a3"
Sep 16 17:25:09.290225 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:09.290044    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c82q" podUID="8cea32ac-e0c2-4c23-8873-acdedd71424b"
Sep 16 17:25:09.290225 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:09.289840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:09.290225 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:09.290170    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:25:10.471941 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.471905    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="e44ef2b3042b7ce52f3955897d2b1b60bbf08a4875cc500e0d2f40dd9416a06f" exitCode=0
Sep 16 17:25:10.472422 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.471983    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"e44ef2b3042b7ce52f3955897d2b1b60bbf08a4875cc500e0d2f40dd9416a06f"}
Sep 16 17:25:10.581867 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.581841    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-164.ec2.internal" event="NodeReady"
Sep 16 17:25:10.582039 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.581997    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 17:25:10.647930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.647883    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:25:10.662678 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.662638    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"]
Sep 16 17:25:10.662831 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.662783    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.673013 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.672983    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.673549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.673529    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 17:25:10.673716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.673699    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jnlm5\""
Sep 16 17:25:10.673792 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.673717    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 17:25:10.673792 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.673785    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 17:25:10.676866 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.676843    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 17:25:10.676979 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.676890    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 17:25:10.677026 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.676992    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xtd7w\""
Sep 16 17:25:10.687230 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.687207    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 17:25:10.694305 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.694266    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"]
Sep 16 17:25:10.705518 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.705487    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zsd8k"]
Sep 16 17:25:10.713788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.713754    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j79n7"]
Sep 16 17:25:10.713971 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.713891    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.717875 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.717852    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 17:25:10.718100 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.717913    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 17:25:10.718161 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.718132    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qld9l\""
Sep 16 17:25:10.721987 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.721930    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:25:10.722110 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.722023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.725328 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.725304    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 17:25:10.725452 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.725333    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 17:25:10.727403 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.727331    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zsd8k"]
Sep 16 17:25:10.728093 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.728070    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dv5nc\""
Sep 16 17:25:10.728327 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.728314    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 17:25:10.735789 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.735761    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j79n7"]
Sep 16 17:25:10.762407 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762407 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762453    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tddw\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b5a31bf-d965-41a6-8e2d-57a5582241a9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.762688 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762628    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762929 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762712    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.762929 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.762776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864005 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.863964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87pp8\" (UniqueName: \"kubernetes.io/projected/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-kube-api-access-87pp8\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.864279    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.864302    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:10.864381 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.864371    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864298    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-tmp-dir\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.864375    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:11.364353173 +0000 UTC m=+34.594134854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864428    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.864459    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:11.364424224 +0000 UTC m=+34.594205894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cmdh\" (UniqueName: \"kubernetes.io/projected/e888ce83-3f2c-4c6a-ab52-23291c51fd24-kube-api-access-9cmdh\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tddw\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-config-volume\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b5a31bf-d965-41a6-8e2d-57a5582241a9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.864665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.865063 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864704    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.865063 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.864732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e888ce83-3f2c-4c6a-ab52-23291c51fd24-cert\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.865233 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.865215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.865290 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.865249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.865355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.865336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b5a31bf-d965-41a6-8e2d-57a5582241a9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:10.868607 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.868584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.868724 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.868590    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.901160 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.901128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.901287 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.901207    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tddw\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:10.965727 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.965687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87pp8\" (UniqueName: \"kubernetes.io/projected/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-kube-api-access-87pp8\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.965727 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.965729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.965962 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.965824    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:10.965962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.965854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-tmp-dir\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.965962 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.965881    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:11.465867067 +0000 UTC m=+34.695648739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:10.965962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.965907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cmdh\" (UniqueName: \"kubernetes.io/projected/e888ce83-3f2c-4c6a-ab52-23291c51fd24-kube-api-access-9cmdh\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.965962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.965936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-config-volume\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.966215 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.966005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:10.966215 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.966028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e888ce83-3f2c-4c6a-ab52-23291c51fd24-cert\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.966215 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.966047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:10.966215 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.966123    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:25:10.966215 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.966172    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret podName:8cea32ac-e0c2-4c23-8873-acdedd71424b nodeName:}" failed. No retries permitted until 2026-09-16 17:25:42.966159534 +0000 UTC m=+66.195941210 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret") pod "global-pull-secret-syncer-7c82q" (UID: "8cea32ac-e0c2-4c23-8873-acdedd71424b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:25:10.966458 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.966214    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:25:10.966458 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:10.966260    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:25:42.966248807 +0000 UTC m=+66.196030481 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:25:10.966458 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.966212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-tmp-dir\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.966458 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.966417    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-config-volume\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.968621 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.968598    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e888ce83-3f2c-4c6a-ab52-23291c51fd24-cert\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:10.987509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.987441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87pp8\" (UniqueName: \"kubernetes.io/projected/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-kube-api-access-87pp8\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:10.987927 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:10.987910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cmdh\" (UniqueName: \"kubernetes.io/projected/e888ce83-3f2c-4c6a-ab52-23291c51fd24-kube-api-access-9cmdh\") pod \"ingress-canary-j79n7\" (UID: \"e888ce83-3f2c-4c6a-ab52-23291c51fd24\") " pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:11.031015 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.030973    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j79n7"
Sep 16 17:25:11.167264 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.167219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:11.167504 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.167387    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:25:11.167504 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.167415    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:25:11.167504 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.167426    2559 projected.go:196] Error preparing data for projected volume kube-api-access-wk85g for pod openshift-network-diagnostics/network-check-target-m7czc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:25:11.167504 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.167488    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g podName:be040180-d5b1-45e6-8907-33e1302ae1a3 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:43.167473842 +0000 UTC m=+66.397255516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wk85g" (UniqueName: "kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g") pod "network-check-target-m7czc" (UID: "be040180-d5b1-45e6-8907-33e1302ae1a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:25:11.289616 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.289503    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:11.289616 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.289546    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:11.289922 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.289643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:11.299229 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.299204    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 17:25:11.299651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.299629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 17:25:11.299742 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.299655    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 17:25:11.299791 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.299739    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp4r4\""
Sep 16 17:25:11.300006 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.299991    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 17:25:11.300061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.300047    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhcq5\""
Sep 16 17:25:11.353811 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.353778    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j79n7"]
Sep 16 17:25:11.358459 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:25:11.358430    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode888ce83_3f2c_4c6a_ab52_23291c51fd24.slice/crio-870895c5485667d626495dcc124ec07ece0d8f5338860ca1b2890be01ced0490 WatchSource:0}: Error finding container 870895c5485667d626495dcc124ec07ece0d8f5338860ca1b2890be01ced0490: Status 404 returned error can't find the container with id 870895c5485667d626495dcc124ec07ece0d8f5338860ca1b2890be01ced0490
Sep 16 17:25:11.369344 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.369311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:11.369508 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.369363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:11.369508 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.369458    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:11.369647 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.369513    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:12.369500048 +0000 UTC m=+35.599281731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:11.369647 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.369511    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:11.369647 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.369534    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:11.369647 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.369613    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:12.369594457 +0000 UTC m=+35.599376138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:11.470366 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.470327    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:11.470673 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.470431    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:11.470673 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:11.470518    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:12.470498219 +0000 UTC m=+35.700279889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:11.477013 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.476984    2559 generic.go:356] "Generic (PLEG): container finished" podID="b70ed42a-ce9e-48b4-a82e-eac803af9cda" containerID="e2da8b38192150f35a29ebae0752e0eee2d1f3ce0ed27c0ddaf5d20311fc0cb1" exitCode=0
Sep 16 17:25:11.477493 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.477068    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerDied","Data":"e2da8b38192150f35a29ebae0752e0eee2d1f3ce0ed27c0ddaf5d20311fc0cb1"}
Sep 16 17:25:11.478289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:11.478267    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j79n7" event={"ID":"e888ce83-3f2c-4c6a-ab52-23291c51fd24","Type":"ContainerStarted","Data":"870895c5485667d626495dcc124ec07ece0d8f5338860ca1b2890be01ced0490"}
Sep 16 17:25:12.380478 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:12.380443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:12.380497    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.380612    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.380628    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.380628    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.380692    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:14.380678556 +0000 UTC m=+37.610460229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:12.380769 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.380707    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:14.380700659 +0000 UTC m=+37.610482327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:12.481238 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:12.481202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:12.481943 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.481355    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:12.481943 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:12.481428    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:14.48140732 +0000 UTC m=+37.711189008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:12.485247 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:12.485221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" event={"ID":"b70ed42a-ce9e-48b4-a82e-eac803af9cda","Type":"ContainerStarted","Data":"eb08d6482c96a1df66553a17f08287425aa856fd19916c69edee4590d03402b0"}
Sep 16 17:25:12.522341 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:12.522289    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6pdp6" podStartSLOduration=7.564249595 podStartE2EDuration="35.522270414s" totalImagesPullingTime="27.958020819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:12.522022324 +0000 UTC m=+35.751804015" watchObservedRunningTime="2026-09-16 17:25:12.522270414 +0000 UTC m=+35.752052105"
Sep 16 17:25:14.399949 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:14.399900    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:14.399969    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.400058    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.400083    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.400116    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.400139    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:18.400124871 +0000 UTC m=+41.629906541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:14.400477 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.400166    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:18.400154101 +0000 UTC m=+41.629935774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:14.492012 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:14.491974    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j79n7" event={"ID":"e888ce83-3f2c-4c6a-ab52-23291c51fd24","Type":"ContainerStarted","Data":"8d5dcf79478f5823bc2d4f2fbbd55b59e1a7d54e27fc79115df53e9b36f4a60c"}
Sep 16 17:25:14.501212 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:14.501185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:14.501349 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.501335    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:14.501400 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:14.501389    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:18.501374175 +0000 UTC m=+41.731155844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:14.523735 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:14.523680    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j79n7" podStartSLOduration=2.098732927 podStartE2EDuration="4.523661814s" totalImagesPullingTime="2.424928887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:14.522637809 +0000 UTC m=+37.752419494" watchObservedRunningTime="2026-09-16 17:25:14.523661814 +0000 UTC m=+37.753443505"
Sep 16 17:25:18.438237 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:18.438032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:18.438261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.438202    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.438315    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.438368    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.438352835 +0000 UTC m=+49.668134504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.438413    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:18.438678 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.438492    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.438442214 +0000 UTC m=+49.668223883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:18.539300 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:18.539256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:18.539487 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.539408    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:18.539487 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:18.539478    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.539460382 +0000 UTC m=+49.769242061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:26.495762 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:26.495715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:26.495815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.495879    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.495902    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.495913    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.495959    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:42.495942432 +0000 UTC m=+65.725724101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:26.496154 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.495974    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:25:42.495967615 +0000 UTC m=+65.725749284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:26.596889 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:26.596841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:26.597111 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.597004    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:26.597111 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:26.597065    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:42.597047609 +0000 UTC m=+65.826829288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:36.891280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:36.891250    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w85dt"
Sep 16 17:25:42.512450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:42.512409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:42.512462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.512584    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.512606    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.512660    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:26:14.512644835 +0000 UTC m=+97.742426509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.512591    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:25:42.512848 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.512718    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:14.512706843 +0000 UTC m=+97.742488512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:25:42.613765 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:42.613737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:25:42.613882 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.613825    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:42.613882 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:42.613865    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:14.613854152 +0000 UTC m=+97.843635821 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:43.016379 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.016336    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:25:43.016379 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.016385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:43.019245 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.019221    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 17:25:43.020185 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.020164    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 17:25:43.027267 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:43.027247    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 17:25:43.027360 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:25:43.027317    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:26:47.027295662 +0000 UTC m=+130.257077366 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : secret "metrics-daemon-secret" not found
Sep 16 17:25:43.030429 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.030406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cea32ac-e0c2-4c23-8873-acdedd71424b-original-pull-secret\") pod \"global-pull-secret-syncer-7c82q\" (UID: \"8cea32ac-e0c2-4c23-8873-acdedd71424b\") " pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:43.106154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.106125    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c82q"
Sep 16 17:25:43.218165 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.218137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:43.221153 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.221134    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 17:25:43.231074 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.231053    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 17:25:43.241524 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.241497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk85g\" (UniqueName: \"kubernetes.io/projected/be040180-d5b1-45e6-8907-33e1302ae1a3-kube-api-access-wk85g\") pod \"network-check-target-m7czc\" (UID: \"be040180-d5b1-45e6-8907-33e1302ae1a3\") " pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:43.376062 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.376040    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7c82q"]
Sep 16 17:25:43.379071 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:25:43.379044    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cea32ac_e0c2_4c23_8873_acdedd71424b.slice/crio-a3a6e6769020a75440ab67b41e97ee6ebf576f3ccc9900807cc1207f1aad2046 WatchSource:0}: Error finding container a3a6e6769020a75440ab67b41e97ee6ebf576f3ccc9900807cc1207f1aad2046: Status 404 returned error can't find the container with id a3a6e6769020a75440ab67b41e97ee6ebf576f3ccc9900807cc1207f1aad2046
Sep 16 17:25:43.418094 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.418071    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhcq5\""
Sep 16 17:25:43.425638 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.425619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:43.562003 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.561976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7c82q" event={"ID":"8cea32ac-e0c2-4c23-8873-acdedd71424b","Type":"ContainerStarted","Data":"a3a6e6769020a75440ab67b41e97ee6ebf576f3ccc9900807cc1207f1aad2046"}
Sep 16 17:25:43.699273 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:43.699217    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7czc"]
Sep 16 17:25:43.702134 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:25:43.702111    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe040180_d5b1_45e6_8907_33e1302ae1a3.slice/crio-2b6f8f09e96bef7a1e4ca6bd1bd5abee78d3f9519d57b969a0774f224a22a21d WatchSource:0}: Error finding container 2b6f8f09e96bef7a1e4ca6bd1bd5abee78d3f9519d57b969a0774f224a22a21d: Status 404 returned error can't find the container with id 2b6f8f09e96bef7a1e4ca6bd1bd5abee78d3f9519d57b969a0774f224a22a21d
Sep 16 17:25:44.566689 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:44.566658    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7czc" event={"ID":"be040180-d5b1-45e6-8907-33e1302ae1a3","Type":"ContainerStarted","Data":"2b6f8f09e96bef7a1e4ca6bd1bd5abee78d3f9519d57b969a0774f224a22a21d"}
Sep 16 17:25:48.579414 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:48.579369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7c82q" event={"ID":"8cea32ac-e0c2-4c23-8873-acdedd71424b","Type":"ContainerStarted","Data":"26d785ed3f3abc999b0349b22f23e504c5e302a93734faaf6e41edfd01eaa0f4"}
Sep 16 17:25:48.580780 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:48.580754    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7czc" event={"ID":"be040180-d5b1-45e6-8907-33e1302ae1a3","Type":"ContainerStarted","Data":"a3d5ff02d5e3ba6f4496137f7e9d8ce8a35c7f17b793545025caa2c8ddfbf997"}
Sep 16 17:25:48.580906 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:48.580880    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:25:48.597658 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:48.597613    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7c82q" podStartSLOduration=66.968245867 podStartE2EDuration="1m11.597599997s" totalImagesPullingTime="4.62935413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:48.596054033 +0000 UTC m=+71.825835725" watchObservedRunningTime="2026-09-16 17:25:48.597599997 +0000 UTC m=+71.827381687"
Sep 16 17:25:48.611178 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:25:48.611138    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m7czc" podStartSLOduration=67.309725383 podStartE2EDuration="1m11.611127893s" totalImagesPullingTime="4.30140251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:48.61100101 +0000 UTC m=+71.840782702" watchObservedRunningTime="2026-09-16 17:25:48.611127893 +0000 UTC m=+71.840909584"
Sep 16 17:26:14.549584 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:26:14.549401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:26:14.549584 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:26:14.549473    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:26:14.549584 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.549570    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:26:14.550166 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.549592    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d47d5d5bb-4mwhj: secret "image-registry-tls" not found
Sep 16 17:26:14.550166 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.549637    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:26:14.550166 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.549657    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls podName:c9c35d00-3e65-4732-9333-0e2c882d4cde nodeName:}" failed. No retries permitted until 2026-09-16 17:27:18.549641693 +0000 UTC m=+161.779423374 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls") pod "image-registry-7d47d5d5bb-4mwhj" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde") : secret "image-registry-tls" not found
Sep 16 17:26:14.550166 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.549703    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:18.54968721 +0000 UTC m=+161.779468888 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:26:14.650270 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:26:14.650232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:26:14.650471 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.650401    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:26:14.650542 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:14.650488    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls podName:4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:18.650466225 +0000 UTC m=+161.880247908 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls") pod "dns-default-zsd8k" (UID: "4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9") : secret "dns-default-metrics-tls" not found
Sep 16 17:26:19.585818 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:26:19.585785    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m7czc"
Sep 16 17:26:47.085805 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:26:47.085742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:26:47.086447 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:47.085939    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 17:26:47.086447 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:26:47.086046    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs podName:d2f284d7-d259-445b-928e-39708d5bb6ef nodeName:}" failed. No retries permitted until 2026-09-16 17:28:49.08602037 +0000 UTC m=+252.315802052 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs") pod "network-metrics-daemon-5vgn8" (UID: "d2f284d7-d259-445b-928e-39708d5bb6ef") : secret "metrics-daemon-secret" not found
Sep 16 17:27:02.905788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.905752    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"]
Sep 16 17:27:02.907433 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.907416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:02.911354 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.911325    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 17:27:02.911354 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.911348    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 17:27:02.914924 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.914904    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 17:27:02.915028 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.914931    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 17:27:02.915028 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.914992    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hrgjf\""
Sep 16 17:27:02.921962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:02.921940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"]
Sep 16 17:27:03.007509 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.007468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w868m\" (UniqueName: \"kubernetes.io/projected/ca70a60f-0ced-4278-a8f9-02518e3dbe26-kube-api-access-w868m\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.007713 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.007535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca70a60f-0ced-4278-a8f9-02518e3dbe26-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.007713 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.007657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.046987 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.046950    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"]
Sep 16 17:27:03.049242 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.049207    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"]
Sep 16 17:27:03.050837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.050813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.052211 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.052136    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"
Sep 16 17:27:03.053841 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.053819    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"]
Sep 16 17:27:03.055822 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.055807    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"
Sep 16 17:27:03.057621 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.057596    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 17:27:03.059848 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.059821    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s5458\""
Sep 16 17:27:03.059970 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.059941    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:27:03.060035 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.059940    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wpnw9\""
Sep 16 17:27:03.060035 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.059976    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 17:27:03.060224 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.060201    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:27:03.060322 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.060275    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9m6nm\""
Sep 16 17:27:03.067148 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.067128    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 17:27:03.081383 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.081355    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"]
Sep 16 17:27:03.084196 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.084159    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"]
Sep 16 17:27:03.086760 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.086732    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"]
Sep 16 17:27:03.108766 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca70a60f-0ced-4278-a8f9-02518e3dbe26-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.108962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108804    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tf5\" (UniqueName: \"kubernetes.io/projected/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-kube-api-access-26tf5\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.108962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d629\" (UniqueName: \"kubernetes.io/projected/4f3b92cb-c442-4365-8905-3138f4365c73-kube-api-access-9d629\") pod \"network-check-source-6f8b768455-vcdhl\" (UID: \"4f3b92cb-c442-4365-8905-3138f4365c73\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"
Sep 16 17:27:03.108962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.108962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108939    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w868m\" (UniqueName: \"kubernetes.io/projected/ca70a60f-0ced-4278-a8f9-02518e3dbe26-kube-api-access-w868m\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.109170 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.109170 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.108987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58dh\" (UniqueName: \"kubernetes.io/projected/c3bb428a-1f57-4e6f-b606-4d60fef7f8b6-kube-api-access-d58dh\") pod \"volume-data-source-validator-86695b7cc7-hgfzz\" (UID: \"c3bb428a-1f57-4e6f-b606-4d60fef7f8b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"
Sep 16 17:27:03.109170 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.109005    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:03.109170 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.109093    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls podName:ca70a60f-0ced-4278-a8f9-02518e3dbe26 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:03.609072509 +0000 UTC m=+146.838854195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhwg8" (UID: "ca70a60f-0ced-4278-a8f9-02518e3dbe26") : secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:03.109665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.109644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca70a60f-0ced-4278-a8f9-02518e3dbe26-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.133429 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.133397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w868m\" (UniqueName: \"kubernetes.io/projected/ca70a60f-0ced-4278-a8f9-02518e3dbe26-kube-api-access-w868m\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.153066 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.153035    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"]
Sep 16 17:27:03.155004 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.154987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.163094 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.163071    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 17:27:03.163227 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.163133    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:27:03.163227 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.163160    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 17:27:03.163828 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.163812    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-99hc4\""
Sep 16 17:27:03.163962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.163947    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 17:27:03.182944 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.182900    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"]
Sep 16 17:27:03.209629 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26tf5\" (UniqueName: \"kubernetes.io/projected/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-kube-api-access-26tf5\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.209629 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca21b5ac-78ca-4cc2-8117-900a72e6f796-serving-cert\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.209834 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209653    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca21b5ac-78ca-4cc2-8117-900a72e6f796-config\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.209834 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d629\" (UniqueName: \"kubernetes.io/projected/4f3b92cb-c442-4365-8905-3138f4365c73-kube-api-access-9d629\") pod \"network-check-source-6f8b768455-vcdhl\" (UID: \"4f3b92cb-c442-4365-8905-3138f4365c73\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"
Sep 16 17:27:03.209834 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.209834 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209793    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58dh\" (UniqueName: \"kubernetes.io/projected/c3bb428a-1f57-4e6f-b606-4d60fef7f8b6-kube-api-access-d58dh\") pod \"volume-data-source-validator-86695b7cc7-hgfzz\" (UID: \"c3bb428a-1f57-4e6f-b606-4d60fef7f8b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"
Sep 16 17:27:03.209959 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.209842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt74g\" (UniqueName: \"kubernetes.io/projected/ca21b5ac-78ca-4cc2-8117-900a72e6f796-kube-api-access-kt74g\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.209959 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.209899    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 17:27:03.209959 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.209956    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls podName:31fc264d-a3d6-4b37-96e5-48ecca7c4c0f nodeName:}" failed. No retries permitted until 2026-09-16 17:27:03.70994112 +0000 UTC m=+146.939722794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-bsfkk" (UID: "31fc264d-a3d6-4b37-96e5-48ecca7c4c0f") : secret "samples-operator-tls" not found
Sep 16 17:27:03.222075 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.222043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58dh\" (UniqueName: \"kubernetes.io/projected/c3bb428a-1f57-4e6f-b606-4d60fef7f8b6-kube-api-access-d58dh\") pod \"volume-data-source-validator-86695b7cc7-hgfzz\" (UID: \"c3bb428a-1f57-4e6f-b606-4d60fef7f8b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"
Sep 16 17:27:03.222946 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.222926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tf5\" (UniqueName: \"kubernetes.io/projected/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-kube-api-access-26tf5\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.223286 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.223270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d629\" (UniqueName: \"kubernetes.io/projected/4f3b92cb-c442-4365-8905-3138f4365c73-kube-api-access-9d629\") pod \"network-check-source-6f8b768455-vcdhl\" (UID: \"4f3b92cb-c442-4365-8905-3138f4365c73\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"
Sep 16 17:27:03.240142 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.240108    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wg9q8"]
Sep 16 17:27:03.242321 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.242284    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.247246 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.247213    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 17:27:03.247399 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.247273    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 17:27:03.248869 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.248776    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 17:27:03.248869 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.248805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:27:03.248869 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.248815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-c9sw2\""
Sep 16 17:27:03.252164 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.252145    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 17:27:03.264824 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.264786    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wg9q8"]
Sep 16 17:27:03.310232 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e742e177-ac18-44c6-8521-7e549c2c1a54-serving-cert\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.310412 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt74g\" (UniqueName: \"kubernetes.io/projected/ca21b5ac-78ca-4cc2-8117-900a72e6f796-kube-api-access-kt74g\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.310412 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310300    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r6pf\" (UniqueName: \"kubernetes.io/projected/e742e177-ac18-44c6-8521-7e549c2c1a54-kube-api-access-7r6pf\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.310412 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca21b5ac-78ca-4cc2-8117-900a72e6f796-config\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.310412 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca21b5ac-78ca-4cc2-8117-900a72e6f796-serving-cert\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.310661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-config\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.310661 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-trusted-ca\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.310921 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.310902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca21b5ac-78ca-4cc2-8117-900a72e6f796-config\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.312719 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.312697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca21b5ac-78ca-4cc2-8117-900a72e6f796-serving-cert\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.320726 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.320699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt74g\" (UniqueName: \"kubernetes.io/projected/ca21b5ac-78ca-4cc2-8117-900a72e6f796-kube-api-access-kt74g\") pod \"service-ca-operator-cf7c776b7-wrgtn\" (UID: \"ca21b5ac-78ca-4cc2-8117-900a72e6f796\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.368517 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.368481    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"
Sep 16 17:27:03.373374 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.373346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"
Sep 16 17:27:03.411946 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.411851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r6pf\" (UniqueName: \"kubernetes.io/projected/e742e177-ac18-44c6-8521-7e549c2c1a54-kube-api-access-7r6pf\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.411946 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.411936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-config\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.412147 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.411997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-trusted-ca\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.412147 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.412049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e742e177-ac18-44c6-8521-7e549c2c1a54-serving-cert\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.413064 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.413004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-config\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.413616 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.413546    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e742e177-ac18-44c6-8521-7e549c2c1a54-trusted-ca\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.415797 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.415717    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e742e177-ac18-44c6-8521-7e549c2c1a54-serving-cert\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.421768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.421677    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r6pf\" (UniqueName: \"kubernetes.io/projected/e742e177-ac18-44c6-8521-7e549c2c1a54-kube-api-access-7r6pf\") pod \"console-operator-696884fb9c-wg9q8\" (UID: \"e742e177-ac18-44c6-8521-7e549c2c1a54\") " pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.463518 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.463486    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"
Sep 16 17:27:03.552178 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.552135    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:03.614133 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.614104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:03.614283 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.614214    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:03.614283 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.614271    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls podName:ca70a60f-0ced-4278-a8f9-02518e3dbe26 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:04.614257049 +0000 UTC m=+147.844038732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhwg8" (UID: "ca70a60f-0ced-4278-a8f9-02518e3dbe26") : secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:03.646098 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.646066    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl"]
Sep 16 17:27:03.650462 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:03.650437    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3b92cb_c442_4365_8905_3138f4365c73.slice/crio-81241f3318d6ed9f9a1288384d2bd06227344f4222d36496cad1021d3d608634 WatchSource:0}: Error finding container 81241f3318d6ed9f9a1288384d2bd06227344f4222d36496cad1021d3d608634: Status 404 returned error can't find the container with id 81241f3318d6ed9f9a1288384d2bd06227344f4222d36496cad1021d3d608634
Sep 16 17:27:03.653406 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.653379    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz"]
Sep 16 17:27:03.661087 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:03.661062    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bb428a_1f57_4e6f_b606_4d60fef7f8b6.slice/crio-a4d9f4848a3a18f4edafc30240083dbbd12844269c46a316139ae3cc96d35c09 WatchSource:0}: Error finding container a4d9f4848a3a18f4edafc30240083dbbd12844269c46a316139ae3cc96d35c09: Status 404 returned error can't find the container with id a4d9f4848a3a18f4edafc30240083dbbd12844269c46a316139ae3cc96d35c09
Sep 16 17:27:03.716413 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.716370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:03.716636 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.716617    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 17:27:03.716709 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:03.716696    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls podName:31fc264d-a3d6-4b37-96e5-48ecca7c4c0f nodeName:}" failed. No retries permitted until 2026-09-16 17:27:04.716669462 +0000 UTC m=+147.946451133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-bsfkk" (UID: "31fc264d-a3d6-4b37-96e5-48ecca7c4c0f") : secret "samples-operator-tls" not found
Sep 16 17:27:03.750826 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:03.750792    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca21b5ac_78ca_4cc2_8117_900a72e6f796.slice/crio-c2e42f811e0e03a2a2167cb35a214dbf50a4a4026f3b5582330730cdb2eb0940 WatchSource:0}: Error finding container c2e42f811e0e03a2a2167cb35a214dbf50a4a4026f3b5582330730cdb2eb0940: Status 404 returned error can't find the container with id c2e42f811e0e03a2a2167cb35a214dbf50a4a4026f3b5582330730cdb2eb0940
Sep 16 17:27:03.755377 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.755354    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn"]
Sep 16 17:27:03.765910 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.765853    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn" event={"ID":"ca21b5ac-78ca-4cc2-8117-900a72e6f796","Type":"ContainerStarted","Data":"c2e42f811e0e03a2a2167cb35a214dbf50a4a4026f3b5582330730cdb2eb0940"}
Sep 16 17:27:03.767425 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.767392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl" event={"ID":"4f3b92cb-c442-4365-8905-3138f4365c73","Type":"ContainerStarted","Data":"5d7cc8eac5b445c1ad26751aa6dd1713420c645346528a3f1005c468865e0595"}
Sep 16 17:27:03.767575 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.767445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl" event={"ID":"4f3b92cb-c442-4365-8905-3138f4365c73","Type":"ContainerStarted","Data":"81241f3318d6ed9f9a1288384d2bd06227344f4222d36496cad1021d3d608634"}
Sep 16 17:27:03.768547 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.768526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz" event={"ID":"c3bb428a-1f57-4e6f-b606-4d60fef7f8b6","Type":"ContainerStarted","Data":"a4d9f4848a3a18f4edafc30240083dbbd12844269c46a316139ae3cc96d35c09"}
Sep 16 17:27:03.789452 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.789389    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-vcdhl" podStartSLOduration=0.789370688 podStartE2EDuration="789.370688ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:03.786046833 +0000 UTC m=+147.015828528" watchObservedRunningTime="2026-09-16 17:27:03.789370688 +0000 UTC m=+147.019152378"
Sep 16 17:27:03.826910 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:03.826878    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wg9q8"]
Sep 16 17:27:03.830397 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:03.830368    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode742e177_ac18_44c6_8521_7e549c2c1a54.slice/crio-be1a195f369dbfb25249eadee1af6484f55249823349d210621ffe4a73b92da1 WatchSource:0}: Error finding container be1a195f369dbfb25249eadee1af6484f55249823349d210621ffe4a73b92da1: Status 404 returned error can't find the container with id be1a195f369dbfb25249eadee1af6484f55249823349d210621ffe4a73b92da1
Sep 16 17:27:04.623934 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:04.623899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:04.624350 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:04.624145    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:04.624350 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:04.624214    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls podName:ca70a60f-0ced-4278-a8f9-02518e3dbe26 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:06.624193324 +0000 UTC m=+149.853975014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhwg8" (UID: "ca70a60f-0ced-4278-a8f9-02518e3dbe26") : secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:04.725445 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:04.725335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:04.725644 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:04.725575    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 17:27:04.725712 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:04.725661    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls podName:31fc264d-a3d6-4b37-96e5-48ecca7c4c0f nodeName:}" failed. No retries permitted until 2026-09-16 17:27:06.72564028 +0000 UTC m=+149.955421963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-bsfkk" (UID: "31fc264d-a3d6-4b37-96e5-48ecca7c4c0f") : secret "samples-operator-tls" not found
Sep 16 17:27:04.774891 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:04.774831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz" event={"ID":"c3bb428a-1f57-4e6f-b606-4d60fef7f8b6","Type":"ContainerStarted","Data":"fb26d195d22c68fd76c716ab54c1a94f02396f0c8e6aa09f952f975a19b66248"}
Sep 16 17:27:04.777795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:04.777759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" event={"ID":"e742e177-ac18-44c6-8521-7e549c2c1a54","Type":"ContainerStarted","Data":"be1a195f369dbfb25249eadee1af6484f55249823349d210621ffe4a73b92da1"}
Sep 16 17:27:04.792937 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:04.792865    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hgfzz" podStartSLOduration=0.877899049 podStartE2EDuration="1.792841472s" totalImagesPullingTime="914.942423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:04.792790559 +0000 UTC m=+148.022572251" watchObservedRunningTime="2026-09-16 17:27:04.792841472 +0000 UTC m=+148.022623165"
Sep 16 17:27:06.641226 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.641189    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:06.641595 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:06.641355    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:06.641595 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:06.641416    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls podName:ca70a60f-0ced-4278-a8f9-02518e3dbe26 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:10.641400651 +0000 UTC m=+153.871182340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhwg8" (UID: "ca70a60f-0ced-4278-a8f9-02518e3dbe26") : secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:06.741978 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.741938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:06.742153 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:06.742092    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 17:27:06.742203 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:06.742156    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls podName:31fc264d-a3d6-4b37-96e5-48ecca7c4c0f nodeName:}" failed. No retries permitted until 2026-09-16 17:27:10.742140742 +0000 UTC m=+153.971922411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-bsfkk" (UID: "31fc264d-a3d6-4b37-96e5-48ecca7c4c0f") : secret "samples-operator-tls" not found
Sep 16 17:27:06.785819 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.785785    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/0.log"
Sep 16 17:27:06.786002 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.785828    2559 generic.go:356] "Generic (PLEG): container finished" podID="e742e177-ac18-44c6-8521-7e549c2c1a54" containerID="22c542ac0bfaaea82b5c1356ec639d9e39ec8e6b03ccf0a97109f8b83a0ef640" exitCode=255
Sep 16 17:27:06.786002 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.785863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" event={"ID":"e742e177-ac18-44c6-8521-7e549c2c1a54","Type":"ContainerDied","Data":"22c542ac0bfaaea82b5c1356ec639d9e39ec8e6b03ccf0a97109f8b83a0ef640"}
Sep 16 17:27:06.786161 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.786141    2559 scope.go:117] "RemoveContainer" containerID="22c542ac0bfaaea82b5c1356ec639d9e39ec8e6b03ccf0a97109f8b83a0ef640"
Sep 16 17:27:06.787804 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.787777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn" event={"ID":"ca21b5ac-78ca-4cc2-8117-900a72e6f796","Type":"ContainerStarted","Data":"1a9241625e0461bcdbfa693f57b832799715a2704886cc37224b00327356d172"}
Sep 16 17:27:06.873723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.873667    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn" podStartSLOduration=1.433147624 podStartE2EDuration="3.873651299s" totalImagesPullingTime="2.440503675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:06.873494029 +0000 UTC m=+150.103275719" watchObservedRunningTime="2026-09-16 17:27:06.873651299 +0000 UTC m=+150.103433028"
Sep 16 17:27:06.881981 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.881940    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"]
Sep 16 17:27:06.883970 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.883945    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"
Sep 16 17:27:06.887474 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.887451    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 16 17:27:06.888072 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.888037    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6tnsd\""
Sep 16 17:27:06.890586 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.888444    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 16 17:27:06.907855 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.907825    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"]
Sep 16 17:27:06.944068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:06.944039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78h2\" (UniqueName: \"kubernetes.io/projected/9018820e-0403-4bcc-bb44-db4942cc15af-kube-api-access-f78h2\") pod \"migrator-8b4d697c5-4pnwk\" (UID: \"9018820e-0403-4bcc-bb44-db4942cc15af\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"
Sep 16 17:27:07.045082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.045021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78h2\" (UniqueName: \"kubernetes.io/projected/9018820e-0403-4bcc-bb44-db4942cc15af-kube-api-access-f78h2\") pod \"migrator-8b4d697c5-4pnwk\" (UID: \"9018820e-0403-4bcc-bb44-db4942cc15af\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"
Sep 16 17:27:07.068169 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.068139    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78h2\" (UniqueName: \"kubernetes.io/projected/9018820e-0403-4bcc-bb44-db4942cc15af-kube-api-access-f78h2\") pod \"migrator-8b4d697c5-4pnwk\" (UID: \"9018820e-0403-4bcc-bb44-db4942cc15af\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"
Sep 16 17:27:07.199429 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.199337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"
Sep 16 17:27:07.390211 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.390176    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qsksl"]
Sep 16 17:27:07.392955 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.392932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.396205 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.396183    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 17:27:07.396325 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.396290    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 17:27:07.396492 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.396480    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 17:27:07.397307 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.397289    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hmpn4\""
Sep 16 17:27:07.397381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.397297    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 17:27:07.417634 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.417606    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qsksl"]
Sep 16 17:27:07.449752 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.449681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acd362b9-234a-43cb-ae75-8bdc130041f2-crio-socket\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.449752 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.449721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.449934 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.449762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acd362b9-234a-43cb-ae75-8bdc130041f2-data-volume\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.449934 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.449787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klslf\" (UniqueName: \"kubernetes.io/projected/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-api-access-klslf\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.449934 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.449820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.474394 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.474368    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk"]
Sep 16 17:27:07.476526 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:07.476496    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9018820e_0403_4bcc_bb44_db4942cc15af.slice/crio-7b5c6cc1955f87186ee356f204e9a1cd801040f41d3fa6856f7a8cb593b00543 WatchSource:0}: Error finding container 7b5c6cc1955f87186ee356f204e9a1cd801040f41d3fa6856f7a8cb593b00543: Status 404 returned error can't find the container with id 7b5c6cc1955f87186ee356f204e9a1cd801040f41d3fa6856f7a8cb593b00543
Sep 16 17:27:07.550313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550278    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acd362b9-234a-43cb-ae75-8bdc130041f2-crio-socket\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acd362b9-234a-43cb-ae75-8bdc130041f2-crio-socket\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:07.550418    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:07.550464    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:08.050451185 +0000 UTC m=+151.280232854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acd362b9-234a-43cb-ae75-8bdc130041f2-data-volume\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550486 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klslf\" (UniqueName: \"kubernetes.io/projected/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-api-access-klslf\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.550799 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:07.550646    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:07.550799 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:07.550683    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:08.050673061 +0000 UTC m=+151.280454735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:07.550917 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.550852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acd362b9-234a-43cb-ae75-8bdc130041f2-data-volume\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.562821 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.562787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klslf\" (UniqueName: \"kubernetes.io/projected/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-api-access-klslf\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:07.793284 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.793208    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:27:07.793703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.793690    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/0.log"
Sep 16 17:27:07.793760 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.793724    2559 generic.go:356] "Generic (PLEG): container finished" podID="e742e177-ac18-44c6-8521-7e549c2c1a54" containerID="9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960" exitCode=255
Sep 16 17:27:07.793796 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.793759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" event={"ID":"e742e177-ac18-44c6-8521-7e549c2c1a54","Type":"ContainerDied","Data":"9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960"}
Sep 16 17:27:07.793853 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.793831    2559 scope.go:117] "RemoveContainer" containerID="22c542ac0bfaaea82b5c1356ec639d9e39ec8e6b03ccf0a97109f8b83a0ef640"
Sep 16 17:27:07.794062 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.794042    2559 scope.go:117] "RemoveContainer" containerID="9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960"
Sep 16 17:27:07.794248 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:07.794228    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wg9q8_openshift-console-operator(e742e177-ac18-44c6-8521-7e549c2c1a54)\"" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" podUID="e742e177-ac18-44c6-8521-7e549c2c1a54"
Sep 16 17:27:07.795109 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:07.795086    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk" event={"ID":"9018820e-0403-4bcc-bb44-db4942cc15af","Type":"ContainerStarted","Data":"7b5c6cc1955f87186ee356f204e9a1cd801040f41d3fa6856f7a8cb593b00543"}
Sep 16 17:27:08.055381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.055286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:08.055381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.055372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:08.055656 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:08.055456    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:08.055656 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:08.055483    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:08.055656 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:08.055537    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:09.055517546 +0000 UTC m=+152.285299215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:08.055656 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:08.055574    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:09.055549022 +0000 UTC m=+152.285330696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:08.799855 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.799767    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:27:08.800300 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.800234    2559 scope.go:117] "RemoveContainer" containerID="9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960"
Sep 16 17:27:08.800453 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:08.800429    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wg9q8_openshift-console-operator(e742e177-ac18-44c6-8521-7e549c2c1a54)\"" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" podUID="e742e177-ac18-44c6-8521-7e549c2c1a54"
Sep 16 17:27:08.801741 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.801713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk" event={"ID":"9018820e-0403-4bcc-bb44-db4942cc15af","Type":"ContainerStarted","Data":"fa4eff7a6e83b427a6dce9142dfd9aa44c2159503701cc4cb2ec303b05d7e9f5"}
Sep 16 17:27:08.801873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.801787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk" event={"ID":"9018820e-0403-4bcc-bb44-db4942cc15af","Type":"ContainerStarted","Data":"24e6aac94316cf555679450bfb488c5b0c2153c22e36454659eb293aea6a8df3"}
Sep 16 17:27:08.838409 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:08.838340    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4pnwk" podStartSLOduration=1.825062959 podStartE2EDuration="2.838319489s" totalImagesPullingTime="1.01325653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:08.838044454 +0000 UTC m=+152.067826144" watchObservedRunningTime="2026-09-16 17:27:08.838319489 +0000 UTC m=+152.068101181"
Sep 16 17:27:09.064231 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.064194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:09.064421 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.064253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:09.064421 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:09.064329    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:09.064421 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:09.064364    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:09.064421 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:09.064382    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:11.064369385 +0000 UTC m=+154.294151067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:09.064594 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:09.064430    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:11.064409611 +0000 UTC m=+154.294191286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:09.606435 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.606388    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j8xt6"]
Sep 16 17:27:09.608750 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.608731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.612135 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.612105    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 17:27:09.612135 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.612123    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 17:27:09.613219 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.613198    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t5f8v\""
Sep 16 17:27:09.613325 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.613199    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 17:27:09.613325 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.613237    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 17:27:09.618504 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.618484    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j8xt6"]
Sep 16 17:27:09.668524 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.668488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-cabundle\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.668732 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.668611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-key\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.668732 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.668683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkvln\" (UniqueName: \"kubernetes.io/projected/d7a45295-0b01-44b1-a80f-5a9c3460b658-kube-api-access-mkvln\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.769750 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.769709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-key\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.769904 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.769817    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkvln\" (UniqueName: \"kubernetes.io/projected/d7a45295-0b01-44b1-a80f-5a9c3460b658-kube-api-access-mkvln\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.769904 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.769850    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-cabundle\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.770580 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.770540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-cabundle\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.772159 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.772142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d7a45295-0b01-44b1-a80f-5a9c3460b658-signing-key\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.779073 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.779049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkvln\" (UniqueName: \"kubernetes.io/projected/d7a45295-0b01-44b1-a80f-5a9c3460b658-kube-api-access-mkvln\") pod \"service-ca-568869fdc5-j8xt6\" (UID: \"d7a45295-0b01-44b1-a80f-5a9c3460b658\") " pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:09.918498 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:09.918398    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-j8xt6"
Sep 16 17:27:10.118816 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.118787    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndng9_1909d2d2-d9ad-4568-a352-55f882e0f6cb/dns-node-resolver/0.log"
Sep 16 17:27:10.196128 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.196035    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j8xt6"]
Sep 16 17:27:10.199174 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:10.199146    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a45295_0b01_44b1_a80f_5a9c3460b658.slice/crio-2913e38769f50d2c7c24f4a91f33a2e62c77ea4565b1790d71792bcda0a061d9 WatchSource:0}: Error finding container 2913e38769f50d2c7c24f4a91f33a2e62c77ea4565b1790d71792bcda0a061d9: Status 404 returned error can't find the container with id 2913e38769f50d2c7c24f4a91f33a2e62c77ea4565b1790d71792bcda0a061d9
Sep 16 17:27:10.677517 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.677475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:10.677718 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:10.677649    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:10.677718 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:10.677714    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls podName:ca70a60f-0ced-4278-a8f9-02518e3dbe26 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:18.677697186 +0000 UTC m=+161.907478878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhwg8" (UID: "ca70a60f-0ced-4278-a8f9-02518e3dbe26") : secret "cluster-monitoring-operator-tls" not found
Sep 16 17:27:10.778860 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.778827    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:10.779029 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:10.778987    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 17:27:10.779080 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:10.779053    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls podName:31fc264d-a3d6-4b37-96e5-48ecca7c4c0f nodeName:}" failed. No retries permitted until 2026-09-16 17:27:18.779037663 +0000 UTC m=+162.008819338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-bsfkk" (UID: "31fc264d-a3d6-4b37-96e5-48ecca7c4c0f") : secret "samples-operator-tls" not found
Sep 16 17:27:10.811309 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.811271    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-j8xt6" event={"ID":"d7a45295-0b01-44b1-a80f-5a9c3460b658","Type":"ContainerStarted","Data":"32503e8df2b0ce28dfc901dde8b0787abf740bac5cff5850c740f6e201be5a7e"}
Sep 16 17:27:10.811477 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.811336    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-j8xt6" event={"ID":"d7a45295-0b01-44b1-a80f-5a9c3460b658","Type":"ContainerStarted","Data":"2913e38769f50d2c7c24f4a91f33a2e62c77ea4565b1790d71792bcda0a061d9"}
Sep 16 17:27:10.835730 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:10.835661    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-j8xt6" podStartSLOduration=1.835636757 podStartE2EDuration="1.835636757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:10.834600081 +0000 UTC m=+154.064381772" watchObservedRunningTime="2026-09-16 17:27:10.835636757 +0000 UTC m=+154.065418450"
Sep 16 17:27:11.082100 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:11.082063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:11.082494 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:11.082121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:11.082494 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:11.082184    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:11.082494 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:11.082214    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:11.082494 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:11.082230    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:15.082217472 +0000 UTC m=+158.311999140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:11.082494 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:11.082266    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:15.082253446 +0000 UTC m=+158.312035120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:11.117553 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:11.117526    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hqchc_b59791de-9fae-4f5f-8a78-5cb47514ee7e/node-ca/0.log"
Sep 16 17:27:11.720722 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:11.720688    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j79n7_e888ce83-3f2c-4c6a-ab52-23291c51fd24/serve-healthcheck-canary/0.log"
Sep 16 17:27:12.320386 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:12.320352    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-4pnwk_9018820e-0403-4bcc-bb44-db4942cc15af/migrator/0.log"
Sep 16 17:27:12.520435 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:12.520408    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-4pnwk_9018820e-0403-4bcc-bb44-db4942cc15af/graceful-termination/0.log"
Sep 16 17:27:13.553067 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.553011    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:13.553466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.553093    2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:13.553466 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.553461    2559 scope.go:117] "RemoveContainer" containerID="9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960"
Sep 16 17:27:13.553669 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:13.553649    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wg9q8_openshift-console-operator(e742e177-ac18-44c6-8521-7e549c2c1a54)\"" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" podUID="e742e177-ac18-44c6-8521-7e549c2c1a54"
Sep 16 17:27:13.688725 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:13.688682    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde"
Sep 16 17:27:13.693771 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:13.693741    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5" podUID="1b5a31bf-d965-41a6-8e2d-57a5582241a9"
Sep 16 17:27:13.723418 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:13.723378    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zsd8k" podUID="4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9"
Sep 16 17:27:13.821838 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.821804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:13.822029 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.821804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:27:13.822029 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:13.821804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:14.307551 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:14.307453    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5vgn8" podUID="d2f284d7-d259-445b-928e-39708d5bb6ef"
Sep 16 17:27:15.121827 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:15.121771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:15.122313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:15.121873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:15.122313 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:15.121943    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:15.122313 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:15.121990    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:15.122313 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:15.122035    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:23.122012093 +0000 UTC m=+166.351793774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:15.122313 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:15.122055    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm podName:acd362b9-234a-43cb-ae75-8bdc130041f2 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:23.122045437 +0000 UTC m=+166.351827108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsksl" (UID: "acd362b9-234a-43cb-ae75-8bdc130041f2") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:18.552378 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.552329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:27:18.552891 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.552428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:18.552891 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:18.552551    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 17:27:18.552891 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:18.552649    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert podName:1b5a31bf-d965-41a6-8e2d-57a5582241a9 nodeName:}" failed. No retries permitted until 2026-09-16 17:29:20.552632852 +0000 UTC m=+283.782414529 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jl5w5" (UID: "1b5a31bf-d965-41a6-8e2d-57a5582241a9") : secret "networking-console-plugin-cert" not found
Sep 16 17:27:18.554918 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.554894    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"image-registry-7d47d5d5bb-4mwhj\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") " pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:18.625823 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.625791    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jnlm5\""
Sep 16 17:27:18.634066 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.634039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:18.653283 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.653250    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:18.655633 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.655615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9-metrics-tls\") pod \"dns-default-zsd8k\" (UID: \"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9\") " pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:18.754051 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.754008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:18.756508 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.756487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca70a60f-0ced-4278-a8f9-02518e3dbe26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhwg8\" (UID: \"ca70a60f-0ced-4278-a8f9-02518e3dbe26\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:18.816243 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.816208    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"
Sep 16 17:27:18.855001 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.854964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:18.857407 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.857381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/31fc264d-a3d6-4b37-96e5-48ecca7c4c0f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-bsfkk\" (UID: \"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:18.911098 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.911058    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:27:18.913970 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:18.913936    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c35d00_3e65_4732_9333_0e2c882d4cde.slice/crio-4fcdeea22d6cdd2809f5c3c90fe0b4b7a63c57ea93616418eedc552a27f3eea4 WatchSource:0}: Error finding container 4fcdeea22d6cdd2809f5c3c90fe0b4b7a63c57ea93616418eedc552a27f3eea4: Status 404 returned error can't find the container with id 4fcdeea22d6cdd2809f5c3c90fe0b4b7a63c57ea93616418eedc552a27f3eea4
Sep 16 17:27:18.925019 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.924996    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qld9l\""
Sep 16 17:27:18.932674 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.932644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:18.962679 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:18.962647    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"
Sep 16 17:27:19.086756 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.086549    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8"]
Sep 16 17:27:19.089827 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:19.089790    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca70a60f_0ced_4278_a8f9_02518e3dbe26.slice/crio-4fd389388109d05ae2288697fcbc0ddaa7d72525c2d4dac004c495f97fdfd719 WatchSource:0}: Error finding container 4fd389388109d05ae2288697fcbc0ddaa7d72525c2d4dac004c495f97fdfd719: Status 404 returned error can't find the container with id 4fd389388109d05ae2288697fcbc0ddaa7d72525c2d4dac004c495f97fdfd719
Sep 16 17:27:19.210933 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.210902    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zsd8k"]
Sep 16 17:27:19.214844 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:19.214813    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5b0bd3_e360_41f1_a3a1_c3da0ef9ffe9.slice/crio-d87712dac27e1c503b5cded6e19742b55653d5a73f77ca309256340fa3a4eaa9 WatchSource:0}: Error finding container d87712dac27e1c503b5cded6e19742b55653d5a73f77ca309256340fa3a4eaa9: Status 404 returned error can't find the container with id d87712dac27e1c503b5cded6e19742b55653d5a73f77ca309256340fa3a4eaa9
Sep 16 17:27:19.255861 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.255827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk"]
Sep 16 17:27:19.846284 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.846236    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8" event={"ID":"ca70a60f-0ced-4278-a8f9-02518e3dbe26","Type":"ContainerStarted","Data":"4fd389388109d05ae2288697fcbc0ddaa7d72525c2d4dac004c495f97fdfd719"}
Sep 16 17:27:19.848086 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.847984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsd8k" event={"ID":"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9","Type":"ContainerStarted","Data":"d87712dac27e1c503b5cded6e19742b55653d5a73f77ca309256340fa3a4eaa9"}
Sep 16 17:27:19.850091 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.849986    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk" event={"ID":"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f","Type":"ContainerStarted","Data":"fe274592dd06398169af6480f29bf396ebbb4aad0714e85935dd0c8740380b56"}
Sep 16 17:27:19.853693 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.853605    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" event={"ID":"c9c35d00-3e65-4732-9333-0e2c882d4cde","Type":"ContainerStarted","Data":"e9566ac2f910f41b17ccdd43a99c3840807fcb27ff184b843a111a71cd359b4b"}
Sep 16 17:27:19.853693 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.853660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" event={"ID":"c9c35d00-3e65-4732-9333-0e2c882d4cde","Type":"ContainerStarted","Data":"4fcdeea22d6cdd2809f5c3c90fe0b4b7a63c57ea93616418eedc552a27f3eea4"}
Sep 16 17:27:19.854023 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.853962    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:19.881618 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:19.880459    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" podStartSLOduration=162.880433969 podStartE2EDuration="2m42.880433969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:19.877510805 +0000 UTC m=+163.107292497" watchObservedRunningTime="2026-09-16 17:27:19.880433969 +0000 UTC m=+163.110215663"
Sep 16 17:27:22.867925 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.867884    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8" event={"ID":"ca70a60f-0ced-4278-a8f9-02518e3dbe26","Type":"ContainerStarted","Data":"06d40c27e42b73b77fc631d9ff8af8b8dd7f82ccb8e35eb4a44ab531b8c02584"}
Sep 16 17:27:22.869783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.869755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsd8k" event={"ID":"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9","Type":"ContainerStarted","Data":"547e486b38e4ef9968961509833fbf63efcf31c0a0a2536a6fc43a43fba06046"}
Sep 16 17:27:22.869942 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.869805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsd8k" event={"ID":"4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9","Type":"ContainerStarted","Data":"a2c6f4614d1dace545a361d32cb5bc94bd9958ac1fccac043c0560ebaa99af6a"}
Sep 16 17:27:22.869942 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.869838    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:22.871567 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.871536    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk" event={"ID":"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f","Type":"ContainerStarted","Data":"a372af56f225c872ad6387319320ff12f70165186827808db20f3511cce3aabd"}
Sep 16 17:27:22.871658 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.871592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk" event={"ID":"31fc264d-a3d6-4b37-96e5-48ecca7c4c0f","Type":"ContainerStarted","Data":"30b05dab3bdc1f54d2f953dcc83a28220370399f23cb875f937fe5182aeef2c4"}
Sep 16 17:27:22.896404 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.896345    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhwg8" podStartSLOduration=18.180099184 podStartE2EDuration="20.896327967s" totalImagesPullingTime="2.716228783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:22.895683622 +0000 UTC m=+166.125465323" watchObservedRunningTime="2026-09-16 17:27:22.896327967 +0000 UTC m=+166.126109658"
Sep 16 17:27:22.916328 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.916278    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-bsfkk" podStartSLOduration=17.40620881 podStartE2EDuration="19.916261492s" totalImagesPullingTime="2.510052682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:22.915332856 +0000 UTC m=+166.145114550" watchObservedRunningTime="2026-09-16 17:27:22.916261492 +0000 UTC m=+166.146043183"
Sep 16 17:27:22.935068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:22.935011    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zsd8k" podStartSLOduration=130.341379708 podStartE2EDuration="2m12.934994718s" totalImagesPullingTime="2.59361501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:22.934467214 +0000 UTC m=+166.164248907" watchObservedRunningTime="2026-09-16 17:27:22.934994718 +0000 UTC m=+166.164776408"
Sep 16 17:27:23.202095 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.201998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:23.202095 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.202055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:23.202683 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.202660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acd362b9-234a-43cb-ae75-8bdc130041f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:23.204347 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.204327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acd362b9-234a-43cb-ae75-8bdc130041f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsksl\" (UID: \"acd362b9-234a-43cb-ae75-8bdc130041f2\") " pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:23.301573 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.301520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qsksl"
Sep 16 17:27:23.578310 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.578278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qsksl"]
Sep 16 17:27:23.581670 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:23.581643    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd362b9_234a_43cb_ae75_8bdc130041f2.slice/crio-26216caa01b24f2fc489807d776d2c0e26cc6a0097d5357f34e4caba52d22084 WatchSource:0}: Error finding container 26216caa01b24f2fc489807d776d2c0e26cc6a0097d5357f34e4caba52d22084: Status 404 returned error can't find the container with id 26216caa01b24f2fc489807d776d2c0e26cc6a0097d5357f34e4caba52d22084
Sep 16 17:27:23.876801 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.876688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsksl" event={"ID":"acd362b9-234a-43cb-ae75-8bdc130041f2","Type":"ContainerStarted","Data":"d8f7a90e74fbb4afac4cd9cbb3c994acb0fa5d148c129a890efa66b2e29a6f7d"}
Sep 16 17:27:23.876801 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:23.876767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsksl" event={"ID":"acd362b9-234a-43cb-ae75-8bdc130041f2","Type":"ContainerStarted","Data":"26216caa01b24f2fc489807d776d2c0e26cc6a0097d5357f34e4caba52d22084"}
Sep 16 17:27:24.882631 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:24.882584    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsksl" event={"ID":"acd362b9-234a-43cb-ae75-8bdc130041f2","Type":"ContainerStarted","Data":"c7820b75dde422c569a8b746a656b25cd446ce70b81af0f5d91f78c8b184d75e"}
Sep 16 17:27:25.887645 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:25.887612    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsksl" event={"ID":"acd362b9-234a-43cb-ae75-8bdc130041f2","Type":"ContainerStarted","Data":"70b537c6efc081a5dedc80dc05e006241b8ab3e5eb6f58dd3012a715bc7d3522"}
Sep 16 17:27:25.909416 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:25.909375    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qsksl" podStartSLOduration=16.921228211 podStartE2EDuration="18.909360006s" totalImagesPullingTime="1.988131795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:27:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:25.907330226 +0000 UTC m=+169.137111916" watchObservedRunningTime="2026-09-16 17:27:25.909360006 +0000 UTC m=+169.139141697"
Sep 16 17:27:26.290357 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:26.290336    2559 scope.go:117] "RemoveContainer" containerID="9aaf0d36d4fe791400a9f3801d73d8b4ec3a433b337a79699dd4c419bc03c960"
Sep 16 17:27:26.892238 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:26.892213    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:27:26.892666 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:26.892339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" event={"ID":"e742e177-ac18-44c6-8521-7e549c2c1a54","Type":"ContainerStarted","Data":"1b89fe21f10813ec2d8f056ccad9085a35805d39993b9a47321440db5f74efde"}
Sep 16 17:27:26.892795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:26.892777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:26.916258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:26.916217    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8" podStartSLOduration=21.558059024 podStartE2EDuration="23.916204229s" totalImagesPullingTime="2.358145205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:26.915975804 +0000 UTC m=+170.145757494" watchObservedRunningTime="2026-09-16 17:27:26.916204229 +0000 UTC m=+170.145985940"
Sep 16 17:27:27.305034 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:27.304963    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-wg9q8"
Sep 16 17:27:28.290479 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:28.290331    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:27:32.879743 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:32.879655    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zsd8k"
Sep 16 17:27:34.596752 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.596723    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"]
Sep 16 17:27:34.600018 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.600000    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.606840 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.606817    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 17:27:34.607812 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.607794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 17:27:34.607888 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.607797    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rl85x\""
Sep 16 17:27:34.608332 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.608316    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 17:27:34.639811 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.639787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"]
Sep 16 17:27:34.787530 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.787505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b86887-846b-4c5f-a7be-91ec234c1385-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.787651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.787534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zmwq\" (UniqueName: \"kubernetes.io/projected/98b86887-846b-4c5f-a7be-91ec234c1385-kube-api-access-7zmwq\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.787651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.787585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.787731 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.787649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.887938 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.887865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.887938 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.887931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b86887-846b-4c5f-a7be-91ec234c1385-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.888085 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.887948    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zmwq\" (UniqueName: \"kubernetes.io/projected/98b86887-846b-4c5f-a7be-91ec234c1385-kube-api-access-7zmwq\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.888085 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.887973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.888663 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.888643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b86887-846b-4c5f-a7be-91ec234c1385-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.890376 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.890355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.890473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.890454    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98b86887-846b-4c5f-a7be-91ec234c1385-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:34.922456 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:34.922432    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zmwq\" (UniqueName: \"kubernetes.io/projected/98b86887-846b-4c5f-a7be-91ec234c1385-kube-api-access-7zmwq\") pod \"prometheus-operator-68bcd7f987-74mzz\" (UID: \"98b86887-846b-4c5f-a7be-91ec234c1385\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:35.208648 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:35.208575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"
Sep 16 17:27:35.487619 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:35.487596    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-74mzz"]
Sep 16 17:27:35.489323 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:35.489298    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b86887_846b_4c5f_a7be_91ec234c1385.slice/crio-a1360ac606ff284122ba520a0100c04c5e0cae790bc738a2c331e5a462af8d37 WatchSource:0}: Error finding container a1360ac606ff284122ba520a0100c04c5e0cae790bc738a2c331e5a462af8d37: Status 404 returned error can't find the container with id a1360ac606ff284122ba520a0100c04c5e0cae790bc738a2c331e5a462af8d37
Sep 16 17:27:35.924316 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:35.924280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz" event={"ID":"98b86887-846b-4c5f-a7be-91ec234c1385","Type":"ContainerStarted","Data":"a1360ac606ff284122ba520a0100c04c5e0cae790bc738a2c331e5a462af8d37"}
Sep 16 17:27:36.929034 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:36.928952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz" event={"ID":"98b86887-846b-4c5f-a7be-91ec234c1385","Type":"ContainerStarted","Data":"54e4b172fc13eb4205b15f217a5c8ea68a3f8d5e8487ae9986ee353d64556db7"}
Sep 16 17:27:36.929034 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:36.929003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz" event={"ID":"98b86887-846b-4c5f-a7be-91ec234c1385","Type":"ContainerStarted","Data":"d8ebf403060918fa2791d06eb72f96cd62d2db694a9ff51584f79581c07f0cf7"}
Sep 16 17:27:36.947256 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:36.947217    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-74mzz" podStartSLOduration=1.858345467 podStartE2EDuration="2.947203891s" totalImagesPullingTime="1.088858424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:36.946080908 +0000 UTC m=+180.175862599" watchObservedRunningTime="2026-09-16 17:27:36.947203891 +0000 UTC m=+180.176985581"
Sep 16 17:27:38.638347 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:38.638313    2559 patch_prober.go:28] interesting pod/image-registry-7d47d5d5bb-4mwhj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 17:27:38.638890 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:38.638370    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:27:39.448182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.448150    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"]
Sep 16 17:27:39.452602 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.452554    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.458837 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.458625    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 17:27:39.459598 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.459575    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wtmts\""
Sep 16 17:27:39.459692 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.459581    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 17:27:39.459692 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.459668    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 17:27:39.522582 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.522549    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"]
Sep 16 17:27:39.547073 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.547047    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kb2wt"]
Sep 16 17:27:39.550382 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.550368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.559082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.559067    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gvkfk\""
Sep 16 17:27:39.559197 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.559182    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 17:27:39.559327 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.559312    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 17:27:39.559327 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.559321    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 17:27:39.621814 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.621790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj26f\" (UniqueName: \"kubernetes.io/projected/72c5e7b4-fa73-485a-992c-232f470fc737-kube-api-access-nj26f\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.621917 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.621824    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.621917 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.621861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72c5e7b4-fa73-485a-992c-232f470fc737-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.622026 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.621939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.622026 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.621996    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.622119 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.622033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.722799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722729    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-sys\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.722799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp96v\" (UniqueName: \"kubernetes.io/projected/44f422e4-8f77-4742-8f5b-0ad0dff980e6-kube-api-access-pp96v\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.722799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72c5e7b4-fa73-485a-992c-232f470fc737-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-wtmp\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-tls\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.722977    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-textfile\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-root\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-metrics-client-ca\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723248 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723680 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj26f\" (UniqueName: \"kubernetes.io/projected/72c5e7b4-fa73-485a-992c-232f470fc737-kube-api-access-nj26f\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723680 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723256    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72c5e7b4-fa73-485a-992c-232f470fc737-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723680 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.723785 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723694    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.723785 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.723742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c5e7b4-fa73-485a-992c-232f470fc737-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.725146 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.725126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.725299 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.725284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72c5e7b4-fa73-485a-992c-232f470fc737-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.742922 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.742898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj26f\" (UniqueName: \"kubernetes.io/projected/72c5e7b4-fa73-485a-992c-232f470fc737-kube-api-access-nj26f\") pod \"kube-state-metrics-59fbdd77d-dq45v\" (UID: \"72c5e7b4-fa73-485a-992c-232f470fc737\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.761747 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.761726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-tls\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-textfile\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-root\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-metrics-client-ca\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-sys\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825327    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp96v\" (UniqueName: \"kubernetes.io/projected/44f422e4-8f77-4742-8f5b-0ad0dff980e6-kube-api-access-pp96v\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-wtmp\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.825611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-wtmp\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.826022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-root\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.826089    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.826365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-textfile\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.826444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f422e4-8f77-4742-8f5b-0ad0dff980e6-sys\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.826996 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.826593    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f422e4-8f77-4742-8f5b-0ad0dff980e6-metrics-client-ca\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.828718 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.828696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.829238 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.829218    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f422e4-8f77-4742-8f5b-0ad0dff980e6-node-exporter-tls\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.842608 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.842589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp96v\" (UniqueName: \"kubernetes.io/projected/44f422e4-8f77-4742-8f5b-0ad0dff980e6-kube-api-access-pp96v\") pod \"node-exporter-kb2wt\" (UID: \"44f422e4-8f77-4742-8f5b-0ad0dff980e6\") " pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.858483 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.858461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kb2wt"
Sep 16 17:27:39.870065 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:39.870038    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f422e4_8f77_4742_8f5b_0ad0dff980e6.slice/crio-f5aec96e4adcc882c92a9441ee11ae63bc15531718216f9fdac89bb40e73ccb4 WatchSource:0}: Error finding container f5aec96e4adcc882c92a9441ee11ae63bc15531718216f9fdac89bb40e73ccb4: Status 404 returned error can't find the container with id f5aec96e4adcc882c92a9441ee11ae63bc15531718216f9fdac89bb40e73ccb4
Sep 16 17:27:39.940225 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:39.940194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kb2wt" event={"ID":"44f422e4-8f77-4742-8f5b-0ad0dff980e6","Type":"ContainerStarted","Data":"f5aec96e4adcc882c92a9441ee11ae63bc15531718216f9fdac89bb40e73ccb4"}
Sep 16 17:27:40.050960 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.050938    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v"]
Sep 16 17:27:40.053237 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:40.053211    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c5e7b4_fa73_485a_992c_232f470fc737.slice/crio-6a7bde6590ae7f3c73f04d7e5a264edec1d81c40cbce2b28a7a9099185a7a44c WatchSource:0}: Error finding container 6a7bde6590ae7f3c73f04d7e5a264edec1d81c40cbce2b28a7a9099185a7a44c: Status 404 returned error can't find the container with id 6a7bde6590ae7f3c73f04d7e5a264edec1d81c40cbce2b28a7a9099185a7a44c
Sep 16 17:27:40.350979 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.350946    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:27:40.355289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.355269    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.357832    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.358059    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.358392    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.358629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.358889    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 17:27:40.359235 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.359067    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 17:27:40.359543 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.359268    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 17:27:40.359543 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.359348    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 17:27:40.359543 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.359459    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 17:27:40.359543 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.359525    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pr7dv\""
Sep 16 17:27:40.376143 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.376118    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:27:40.430296 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430296 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430296 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc94l\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430296 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430296 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.430716 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.430606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531615 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc94l\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.531783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531880    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532036 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.531950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532303 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.532015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.532303 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:40.532195    2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 16 17:27:40.532303 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:27:40.532253    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls podName:bc3aed0a-6861-4498-852d-76111b24e185 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:41.032235579 +0000 UTC m=+184.262017262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "bc3aed0a-6861-4498-852d-76111b24e185") : secret "alertmanager-main-tls" not found
Sep 16 17:27:40.535343 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.533603    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.535343 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.534372    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.535343 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.535115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.535573 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.535538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.536353 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.536311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.538102 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.538029    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.538999 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.538694    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.539341 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.539300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.539662 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.539640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.539816 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.539770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.545976 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.545951    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.548143 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.548120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc94l\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:40.863352 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.863321    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:27:40.944938 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:40.944902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v" event={"ID":"72c5e7b4-fa73-485a-992c-232f470fc737","Type":"ContainerStarted","Data":"6a7bde6590ae7f3c73f04d7e5a264edec1d81c40cbce2b28a7a9099185a7a44c"}
Sep 16 17:27:41.036810 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.036775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:41.039442 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.039417    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:41.268013 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.267989    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:27:41.563920 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.563896    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:27:41.564992 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:41.564972    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3aed0a_6861_4498_852d_76111b24e185.slice/crio-24cf55ddb02dc0b817eee0e161eac662d2b28f19f639cb0be04589fc7afab311 WatchSource:0}: Error finding container 24cf55ddb02dc0b817eee0e161eac662d2b28f19f639cb0be04589fc7afab311: Status 404 returned error can't find the container with id 24cf55ddb02dc0b817eee0e161eac662d2b28f19f639cb0be04589fc7afab311
Sep 16 17:27:41.952608 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.952572    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v" event={"ID":"72c5e7b4-fa73-485a-992c-232f470fc737","Type":"ContainerStarted","Data":"275a50e6621314dd6ae4bb099aaa22e4a1b5f7912504ccbe756ce84047c1d0c5"}
Sep 16 17:27:41.953069 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.952642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v" event={"ID":"72c5e7b4-fa73-485a-992c-232f470fc737","Type":"ContainerStarted","Data":"09d53dc27c5846edd5ce3f9348c3a35c30dd408c7576c28bf15c64b990b75e39"}
Sep 16 17:27:41.953069 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.952679    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v" event={"ID":"72c5e7b4-fa73-485a-992c-232f470fc737","Type":"ContainerStarted","Data":"aa70ba6207eccf073a963b870c672ec7727d45541d3ecf0ddd34b56862981cc6"}
Sep 16 17:27:41.954866 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.954836    2559 generic.go:356] "Generic (PLEG): container finished" podID="44f422e4-8f77-4742-8f5b-0ad0dff980e6" containerID="7c4a6f7edd2527391390774af10bd6b086624e50211df306cac3a7dce1f0cc47" exitCode=0
Sep 16 17:27:41.955080 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.954918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kb2wt" event={"ID":"44f422e4-8f77-4742-8f5b-0ad0dff980e6","Type":"ContainerDied","Data":"7c4a6f7edd2527391390774af10bd6b086624e50211df306cac3a7dce1f0cc47"}
Sep 16 17:27:41.956824 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.956797    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"24cf55ddb02dc0b817eee0e161eac662d2b28f19f639cb0be04589fc7afab311"}
Sep 16 17:27:41.973283 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:41.973234    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-dq45v" podStartSLOduration=1.769208323 podStartE2EDuration="2.973214782s" totalImagesPullingTime="1.204006459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:41.971892348 +0000 UTC m=+185.201674060" watchObservedRunningTime="2026-09-16 17:27:41.973214782 +0000 UTC m=+185.202996494"
Sep 16 17:27:42.962289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:42.962256    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kb2wt" event={"ID":"44f422e4-8f77-4742-8f5b-0ad0dff980e6","Type":"ContainerStarted","Data":"37d0a923ed980c449cdc5a7f8e1cbf30a4691f9786b56cf4692484a9237a2988"}
Sep 16 17:27:42.962765 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:42.962342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kb2wt" event={"ID":"44f422e4-8f77-4742-8f5b-0ad0dff980e6","Type":"ContainerStarted","Data":"09952b24a02323903ffb73f2341053bfb89efb87d4d29f46de945c0caaadd6c1"}
Sep 16 17:27:42.963704 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:42.963678    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762" exitCode=0
Sep 16 17:27:42.963818 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:42.963754    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"}
Sep 16 17:27:42.987336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:42.987298    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kb2wt" podStartSLOduration=2.528410127 podStartE2EDuration="3.987282366s" totalImagesPullingTime="1.383618057s" totalInitContainerRuntime="75.254182ms" isStatefulPod=true podCreationTimestamp="2026-09-16 17:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:42.986016786 +0000 UTC m=+186.215798493" watchObservedRunningTime="2026-09-16 17:27:42.987282366 +0000 UTC m=+186.217064058"
Sep 16 17:27:44.134479 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.134447    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56d6969896-gts96"]
Sep 16 17:27:44.138373 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.138344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.142197 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.142174    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-43rcuvshivnpi\""
Sep 16 17:27:44.142288 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.142218    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 17:27:44.143250 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.143058    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 17:27:44.143990 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.143826    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4mj5t\""
Sep 16 17:27:44.143990 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.143946    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 17:27:44.144520 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.144363    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 17:27:44.145071 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.145020    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56d6969896-gts96"]
Sep 16 17:27:44.166440 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/020d37a4-e265-4359-b111-a5373aa2f21e-audit-log\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166590 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-client-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166590 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-metrics-server-audit-profiles\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166677 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-client-certs\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166711 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166674    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc9tl\" (UniqueName: \"kubernetes.io/projected/020d37a4-e265-4359-b111-a5373aa2f21e-kube-api-access-lc9tl\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166763 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.166813 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.166778    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-tls\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.267972 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.267928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-metrics-server-audit-profiles\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268141 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-client-certs\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268141 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc9tl\" (UniqueName: \"kubernetes.io/projected/020d37a4-e265-4359-b111-a5373aa2f21e-kube-api-access-lc9tl\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268141 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268125    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268301 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-tls\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268478 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/020d37a4-e265-4359-b111-a5373aa2f21e-audit-log\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268588 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-client-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268834 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.268912 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.268813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/020d37a4-e265-4359-b111-a5373aa2f21e-audit-log\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.269124 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.269078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/020d37a4-e265-4359-b111-a5373aa2f21e-metrics-server-audit-profiles\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.271138 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.271118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-tls\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.271368 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.271338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-client-ca-bundle\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.271920 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.271898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/020d37a4-e265-4359-b111-a5373aa2f21e-secret-metrics-server-client-certs\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.278388 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.278371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc9tl\" (UniqueName: \"kubernetes.io/projected/020d37a4-e265-4359-b111-a5373aa2f21e-kube-api-access-lc9tl\") pod \"metrics-server-56d6969896-gts96\" (UID: \"020d37a4-e265-4359-b111-a5373aa2f21e\") " pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.451328 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.451298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:27:44.747601 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.744483    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56d6969896-gts96"]
Sep 16 17:27:44.747601 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:44.747351    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020d37a4_e265_4359_b111_a5373aa2f21e.slice/crio-49fb544a2130c5413a4fd762feb19cfa4ea97d22fe3eb56ba6d3bf8da4c5cf4c WatchSource:0}: Error finding container 49fb544a2130c5413a4fd762feb19cfa4ea97d22fe3eb56ba6d3bf8da4c5cf4c: Status 404 returned error can't find the container with id 49fb544a2130c5413a4fd762feb19cfa4ea97d22fe3eb56ba6d3bf8da4c5cf4c
Sep 16 17:27:44.973418 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.973323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56d6969896-gts96" event={"ID":"020d37a4-e265-4359-b111-a5373aa2f21e","Type":"ContainerStarted","Data":"49fb544a2130c5413a4fd762feb19cfa4ea97d22fe3eb56ba6d3bf8da4c5cf4c"}
Sep 16 17:27:44.976381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.976357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"}
Sep 16 17:27:44.976485 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.976418    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"}
Sep 16 17:27:44.976485 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.976445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"}
Sep 16 17:27:44.976485 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.976469    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"}
Sep 16 17:27:44.976612 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:44.976491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"}
Sep 16 17:27:45.533109 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.533067    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:27:45.538981 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.538949    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.543611 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.543544    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 17:27:45.546255 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.545188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 17:27:45.547759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.547266    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 17:27:45.547759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.547510    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 17:27:45.547759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.547543    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 17:27:45.547759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.547636    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3lsd4ju8kvk6k\""
Sep 16 17:27:45.548103 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.547864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 17:27:45.548705 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.548653    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 17:27:45.548891 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.548869    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 17:27:45.549825 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.549802    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 17:27:45.550128 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.550108    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 17:27:45.550541 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.550341    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 17:27:45.550738 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.550570    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zhmxg\""
Sep 16 17:27:45.551786 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.550847    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 17:27:45.551786 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.551648    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 17:27:45.553816 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.553676    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 17:27:45.556524 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.556430    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580425    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580589    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580669    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580716    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580751    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.580930 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.580825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.581757 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.581235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55phv\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.681975 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.681934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.681975 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.681991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682078    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682258 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55phv\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682576    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682601    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.682714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.682625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.683367 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.683260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.684367 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.684313    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.685020 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.684957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.685130 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.685035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.685249 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.685223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.686261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.685669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.686261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.685914    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.686261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.686198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.687081 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.686977    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.687169 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.687033    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.687398 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.687351    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.687473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.687408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.688198 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.688168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.689043 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.689018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.689132 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.689072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.689378 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.689333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.689480 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.689456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.690030 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.690003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.690755 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.690731    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.695542 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.695509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55phv\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv\") pod \"prometheus-k8s-0\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.859410 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.859344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:27:45.986162 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:45.986125    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerStarted","Data":"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"}
Sep 16 17:27:46.022583 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.022492    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.87971293 podStartE2EDuration="6.022462333s" totalImagesPullingTime="3.126674989s" totalInitContainerRuntime="16.074414ms" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:46.019627123 +0000 UTC m=+189.249408817" watchObservedRunningTime="2026-09-16 17:27:46.022462333 +0000 UTC m=+189.252244025"
Sep 16 17:27:46.461292 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.461263    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:27:46.463216 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:27:46.463186    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2888896_d825_4511_8541_27adaa7f4306.slice/crio-d02afd79e52ca31821189b47c824b20d0e412b31dde8dbeac54e256504b69821 WatchSource:0}: Error finding container d02afd79e52ca31821189b47c824b20d0e412b31dde8dbeac54e256504b69821: Status 404 returned error can't find the container with id d02afd79e52ca31821189b47c824b20d0e412b31dde8dbeac54e256504b69821
Sep 16 17:27:46.992902 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.992858    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69" exitCode=0
Sep 16 17:27:46.993391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.992940    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"}
Sep 16 17:27:46.993391 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.993029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"d02afd79e52ca31821189b47c824b20d0e412b31dde8dbeac54e256504b69821"}
Sep 16 17:27:46.995194 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:46.995162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56d6969896-gts96" event={"ID":"020d37a4-e265-4359-b111-a5373aa2f21e","Type":"ContainerStarted","Data":"4aae1cdafba2aeb52e1f3155a99c607a16efc44a3767f9b796ea70fd4dc316e1"}
Sep 16 17:27:47.063603 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:47.063525    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56d6969896-gts96" podStartSLOduration=1.600596946 podStartE2EDuration="3.063505392s" totalImagesPullingTime="1.462908446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:47.062547553 +0000 UTC m=+190.292329244" watchObservedRunningTime="2026-09-16 17:27:47.063505392 +0000 UTC m=+190.293287148"
Sep 16 17:27:51.016547 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:51.016501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"}
Sep 16 17:27:51.017048 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:51.016619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"}
Sep 16 17:27:53.029150 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.029073    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"}
Sep 16 17:27:53.029150 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.029139    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"}
Sep 16 17:27:53.029553 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.029166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"}
Sep 16 17:27:53.029553 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.029203    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerStarted","Data":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"}
Sep 16 17:27:53.065784 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.065707    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.708606086 podStartE2EDuration="8.065686565s" totalImagesPullingTime="5.341902502s" totalInitContainerRuntime="15.177977ms" isStatefulPod=false podCreationTimestamp="2026-09-16 17:27:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:53.062283566 +0000 UTC m=+196.292065295" watchObservedRunningTime="2026-09-16 17:27:53.065686565 +0000 UTC m=+196.295468286"
Sep 16 17:27:53.888363 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:53.888328    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:27:55.859965 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:27:55.859935    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:28:04.451877 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:04.451844    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:28:04.452236 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:04.451923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:28:18.909325 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:18.909253    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerName="registry" containerID="cri-o://e9566ac2f910f41b17ccdd43a99c3840807fcb27ff184b843a111a71cd359b4b" gracePeriod=30
Sep 16 17:28:19.135740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.135707    2559 generic.go:356] "Generic (PLEG): container finished" podID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerID="e9566ac2f910f41b17ccdd43a99c3840807fcb27ff184b843a111a71cd359b4b" exitCode=0
Sep 16 17:28:19.135929 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.135783    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" event={"ID":"c9c35d00-3e65-4732-9333-0e2c882d4cde","Type":"ContainerDied","Data":"e9566ac2f910f41b17ccdd43a99c3840807fcb27ff184b843a111a71cd359b4b"}
Sep 16 17:28:19.161051 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.160984    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210052    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tddw\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210149    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210177    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210236    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210274    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210310    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.211364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.210339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration\") pod \"c9c35d00-3e65-4732-9333-0e2c882d4cde\" (UID: \"c9c35d00-3e65-4732-9333-0e2c882d4cde\") "
Sep 16 17:28:19.215698 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.212911    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:28:19.215698 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.213544    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:28:19.216684 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.216158    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:28:19.216817 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.216789    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:28:19.216998 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.216969    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:28:19.217134 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.217110    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw" (OuterVolumeSpecName: "kube-api-access-9tddw") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "kube-api-access-9tddw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:28:19.217227 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.217141    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:28:19.222791 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.222766    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c9c35d00-3e65-4732-9333-0e2c882d4cde" (UID: "c9c35d00-3e65-4732-9333-0e2c882d4cde"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:28:19.311536 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311502    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-bound-sa-token\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311536 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311529    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-trusted-ca\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311536 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311538    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9c35d00-3e65-4732-9333-0e2c882d4cde-ca-trust-extracted\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311547    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-tls\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311577    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-image-registry-private-configuration\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311588    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9c35d00-3e65-4732-9333-0e2c882d4cde-registry-certificates\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311597    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tddw\" (UniqueName: \"kubernetes.io/projected/c9c35d00-3e65-4732-9333-0e2c882d4cde-kube-api-access-9tddw\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:19.311768 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:19.311606    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9c35d00-3e65-4732-9333-0e2c882d4cde-installation-pull-secrets\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:28:20.141394 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:20.141362    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"
Sep 16 17:28:20.141394 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:20.141374    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d47d5d5bb-4mwhj" event={"ID":"c9c35d00-3e65-4732-9333-0e2c882d4cde","Type":"ContainerDied","Data":"4fcdeea22d6cdd2809f5c3c90fe0b4b7a63c57ea93616418eedc552a27f3eea4"}
Sep 16 17:28:20.141939 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:20.141451    2559 scope.go:117] "RemoveContainer" containerID="e9566ac2f910f41b17ccdd43a99c3840807fcb27ff184b843a111a71cd359b4b"
Sep 16 17:28:20.159779 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:20.159756    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:28:20.163983 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:20.163961    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d47d5d5bb-4mwhj"]
Sep 16 17:28:21.295699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:21.295665    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" path="/var/lib/kubelet/pods/c9c35d00-3e65-4732-9333-0e2c882d4cde/volumes"
Sep 16 17:28:24.457123 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:24.457091    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:28:24.461058 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:24.461032    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56d6969896-gts96"
Sep 16 17:28:32.195164 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:32.195127    2559 generic.go:356] "Generic (PLEG): container finished" podID="ca21b5ac-78ca-4cc2-8117-900a72e6f796" containerID="1a9241625e0461bcdbfa693f57b832799715a2704886cc37224b00327356d172" exitCode=0
Sep 16 17:28:32.195548 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:32.195205    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn" event={"ID":"ca21b5ac-78ca-4cc2-8117-900a72e6f796","Type":"ContainerDied","Data":"1a9241625e0461bcdbfa693f57b832799715a2704886cc37224b00327356d172"}
Sep 16 17:28:32.195548 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:32.195509    2559 scope.go:117] "RemoveContainer" containerID="1a9241625e0461bcdbfa693f57b832799715a2704886cc37224b00327356d172"
Sep 16 17:28:33.202481 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:33.202427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wrgtn" event={"ID":"ca21b5ac-78ca-4cc2-8117-900a72e6f796","Type":"ContainerStarted","Data":"f7d521d227e9f9229c6130c452fce4a7b27ad9becbecd0911219e60279b7ee4f"}
Sep 16 17:28:45.859735 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:45.859689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:28:45.879145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:45.879116    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:28:46.274629 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:46.274539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:28:49.086963 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:49.086925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:28:49.089414 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:49.089385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f284d7-d259-445b-928e-39708d5bb6ef-metrics-certs\") pod \"network-metrics-daemon-5vgn8\" (UID: \"d2f284d7-d259-445b-928e-39708d5bb6ef\") " pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:28:49.295043 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:49.295014    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp4r4\""
Sep 16 17:28:49.302471 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:49.302435    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vgn8"
Sep 16 17:28:49.585694 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:49.585665    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5vgn8"]
Sep 16 17:28:49.588938 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:28:49.588912    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f284d7_d259_445b_928e_39708d5bb6ef.slice/crio-34c0d9c1dfa906ba9966af5ee490b1a56ccff0c8793817482d3b4b56c841640d WatchSource:0}: Error finding container 34c0d9c1dfa906ba9966af5ee490b1a56ccff0c8793817482d3b4b56c841640d: Status 404 returned error can't find the container with id 34c0d9c1dfa906ba9966af5ee490b1a56ccff0c8793817482d3b4b56c841640d
Sep 16 17:28:50.278995 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:50.278957    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vgn8" event={"ID":"d2f284d7-d259-445b-928e-39708d5bb6ef","Type":"ContainerStarted","Data":"34c0d9c1dfa906ba9966af5ee490b1a56ccff0c8793817482d3b4b56c841640d"}
Sep 16 17:28:51.284747 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:51.284712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vgn8" event={"ID":"d2f284d7-d259-445b-928e-39708d5bb6ef","Type":"ContainerStarted","Data":"f3ea110ec3b09d6e391c56e6987934222e5e75fc2c6fcf1e8ac0ae76540f2761"}
Sep 16 17:28:51.285186 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:51.284766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vgn8" event={"ID":"d2f284d7-d259-445b-928e-39708d5bb6ef","Type":"ContainerStarted","Data":"38f5c4c4c164c91f1fb162afbf7ca5cfb1e0ada31ee6aa2e0cdd5d186c73dd0a"}
Sep 16 17:28:51.305251 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:51.305189    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5vgn8" podStartSLOduration=253.495714904 podStartE2EDuration="4m14.305169772s" totalImagesPullingTime="809.454868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:28:51.303033499 +0000 UTC m=+254.532815198" watchObservedRunningTime="2026-09-16 17:28:51.305169772 +0000 UTC m=+254.534951464"
Sep 16 17:28:59.652173 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652084    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652371    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="alertmanager" containerID="cri-o://31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10" gracePeriod=120
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652442    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-metric" containerID="cri-o://d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071" gracePeriod=120
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652478    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="prom-label-proxy" containerID="cri-o://b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5" gracePeriod=120
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652518    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy" containerID="cri-o://89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d" gracePeriod=120
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652533    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="config-reloader" containerID="cri-o://0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049" gracePeriod=120
Sep 16 17:28:59.652699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:28:59.652506    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-web" containerID="cri-o://c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254" gracePeriod=120
Sep 16 17:29:00.322727 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322692    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5" exitCode=0
Sep 16 17:29:00.322727 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322719    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d" exitCode=0
Sep 16 17:29:00.322727 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322729    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049" exitCode=0
Sep 16 17:29:00.322974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322738    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10" exitCode=0
Sep 16 17:29:00.322974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"}
Sep 16 17:29:00.322974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"}
Sep 16 17:29:00.322974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322877    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"}
Sep 16 17:29:00.322974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.322901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"}
Sep 16 17:29:00.890335 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.890304    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:00.997920 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997818    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc94l\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.997920 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.997920 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997963    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.997994    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998022    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998047    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998126    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998195 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998234    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bc3aed0a-6861-4498-852d-76111b24e185\" (UID: \"bc3aed0a-6861-4498-852d-76111b24e185\") "
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998278    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998433    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998670    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998698    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-main-db\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:00.998723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:00.998721    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.001315 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001272    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume" (OuterVolumeSpecName: "config-volume") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.001682 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001652    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l" (OuterVolumeSpecName: "kube-api-access-cc94l") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "kube-api-access-cc94l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:29:01.001682 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001660    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.001856 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001739    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.001856 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.001970 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.001921    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out" (OuterVolumeSpecName: "config-out") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:29:01.002419 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.002383    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.002808 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.002789    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:29:01.005690 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.005462    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.013084 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.013053    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config" (OuterVolumeSpecName: "web-config") pod "bc3aed0a-6861-4498-852d-76111b24e185" (UID: "bc3aed0a-6861-4498-852d-76111b24e185"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:01.099355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099302    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099349    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc3aed0a-6861-4498-852d-76111b24e185-metrics-client-ca\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099359    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc3aed0a-6861-4498-852d-76111b24e185-config-out\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099370    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-tls-assets\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099379    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099388    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc94l\" (UniqueName: \"kubernetes.io/projected/bc3aed0a-6861-4498-852d-76111b24e185-kube-api-access-cc94l\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099399    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-main-tls\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099407    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099416    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-config-volume\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099424    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-cluster-tls-config\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.099643 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.099432    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc3aed0a-6861-4498-852d-76111b24e185-web-config\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:01.330453 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330403    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071" exitCode=0
Sep 16 17:29:01.330453 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330430    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc3aed0a-6861-4498-852d-76111b24e185" containerID="c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254" exitCode=0
Sep 16 17:29:01.330702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"}
Sep 16 17:29:01.330702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330487    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.330702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"}
Sep 16 17:29:01.330702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bc3aed0a-6861-4498-852d-76111b24e185","Type":"ContainerDied","Data":"24cf55ddb02dc0b817eee0e161eac662d2b28f19f639cb0be04589fc7afab311"}
Sep 16 17:29:01.330702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.330649    2559 scope.go:117] "RemoveContainer" containerID="b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"
Sep 16 17:29:01.338712 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.338688    2559 scope.go:117] "RemoveContainer" containerID="d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"
Sep 16 17:29:01.346381 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.346359    2559 scope.go:117] "RemoveContainer" containerID="89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"
Sep 16 17:29:01.353841 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.353782    2559 scope.go:117] "RemoveContainer" containerID="c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"
Sep 16 17:29:01.357940 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.357903    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:29:01.360599 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.360538    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:29:01.363921 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.363903    2559 scope.go:117] "RemoveContainer" containerID="0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"
Sep 16 17:29:01.371895 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.371875    2559 scope.go:117] "RemoveContainer" containerID="31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"
Sep 16 17:29:01.379758 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.379737    2559 scope.go:117] "RemoveContainer" containerID="082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"
Sep 16 17:29:01.387788 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.387763    2559 scope.go:117] "RemoveContainer" containerID="b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"
Sep 16 17:29:01.388107 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.388086    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5\": container with ID starting with b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5 not found: ID does not exist" containerID="b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"
Sep 16 17:29:01.388191 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388122    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"} err="failed to get container status \"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5\": rpc error: code = NotFound desc = could not find container \"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5\": container with ID starting with b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5 not found: ID does not exist"
Sep 16 17:29:01.388191 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388169    2559 scope.go:117] "RemoveContainer" containerID="d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"
Sep 16 17:29:01.388455 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.388439    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071\": container with ID starting with d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071 not found: ID does not exist" containerID="d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"
Sep 16 17:29:01.388523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388461    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"} err="failed to get container status \"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071\": rpc error: code = NotFound desc = could not find container \"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071\": container with ID starting with d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071 not found: ID does not exist"
Sep 16 17:29:01.388523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388477    2559 scope.go:117] "RemoveContainer" containerID="89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"
Sep 16 17:29:01.388730 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.388716    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d\": container with ID starting with 89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d not found: ID does not exist" containerID="89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"
Sep 16 17:29:01.388774 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388733    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"} err="failed to get container status \"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d\": rpc error: code = NotFound desc = could not find container \"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d\": container with ID starting with 89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d not found: ID does not exist"
Sep 16 17:29:01.388774 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388750    2559 scope.go:117] "RemoveContainer" containerID="c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"
Sep 16 17:29:01.388973 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.388952    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254\": container with ID starting with c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254 not found: ID does not exist" containerID="c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"
Sep 16 17:29:01.389042 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.388981    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"} err="failed to get container status \"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254\": rpc error: code = NotFound desc = could not find container \"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254\": container with ID starting with c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254 not found: ID does not exist"
Sep 16 17:29:01.389042 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389001    2559 scope.go:117] "RemoveContainer" containerID="0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"
Sep 16 17:29:01.389225 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.389211    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049\": container with ID starting with 0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049 not found: ID does not exist" containerID="0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"
Sep 16 17:29:01.389259 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389231    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"} err="failed to get container status \"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049\": rpc error: code = NotFound desc = could not find container \"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049\": container with ID starting with 0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049 not found: ID does not exist"
Sep 16 17:29:01.389259 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389244    2559 scope.go:117] "RemoveContainer" containerID="31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"
Sep 16 17:29:01.389450 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.389435    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10\": container with ID starting with 31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10 not found: ID does not exist" containerID="31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"
Sep 16 17:29:01.389487 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389451    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"} err="failed to get container status \"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10\": rpc error: code = NotFound desc = could not find container \"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10\": container with ID starting with 31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10 not found: ID does not exist"
Sep 16 17:29:01.389487 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389463    2559 scope.go:117] "RemoveContainer" containerID="082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"
Sep 16 17:29:01.389670 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:01.389652    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762\": container with ID starting with 082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762 not found: ID does not exist" containerID="082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"
Sep 16 17:29:01.389721 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389674    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"} err="failed to get container status \"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762\": rpc error: code = NotFound desc = could not find container \"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762\": container with ID starting with 082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762 not found: ID does not exist"
Sep 16 17:29:01.389721 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389686    2559 scope.go:117] "RemoveContainer" containerID="b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"
Sep 16 17:29:01.389962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389947    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5"} err="failed to get container status \"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5\": rpc error: code = NotFound desc = could not find container \"b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5\": container with ID starting with b2aad5134a19bbe0875b3bf3a5abb89e9c62367f495bcaadbb1c8b29c8cf2eb5 not found: ID does not exist"
Sep 16 17:29:01.389962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.389962    2559 scope.go:117] "RemoveContainer" containerID="d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"
Sep 16 17:29:01.390200 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390181    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071"} err="failed to get container status \"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071\": rpc error: code = NotFound desc = could not find container \"d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071\": container with ID starting with d3bdafe9729a52ef2913a8209d5924b2246ece8bf4e1cc5acf7a722619a55071 not found: ID does not exist"
Sep 16 17:29:01.390261 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390200    2559 scope.go:117] "RemoveContainer" containerID="89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"
Sep 16 17:29:01.390460 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390433    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d"} err="failed to get container status \"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d\": rpc error: code = NotFound desc = could not find container \"89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d\": container with ID starting with 89ba81a7d52e4520d99fe9e633be1b90822a2795e660250b4649f3b50de4c30d not found: ID does not exist"
Sep 16 17:29:01.390506 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390462    2559 scope.go:117] "RemoveContainer" containerID="c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"
Sep 16 17:29:01.390742 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390724    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254"} err="failed to get container status \"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254\": rpc error: code = NotFound desc = could not find container \"c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254\": container with ID starting with c1804580c7d3abd59254d6b2b0ef19ed34b75476dee65107ae15b4363826e254 not found: ID does not exist"
Sep 16 17:29:01.390802 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390743    2559 scope.go:117] "RemoveContainer" containerID="0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"
Sep 16 17:29:01.390959 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390942    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049"} err="failed to get container status \"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049\": rpc error: code = NotFound desc = could not find container \"0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049\": container with ID starting with 0ed61cc73645cee8d9c3f83b517169eaa465330846d1ca56ffadfac551209049 not found: ID does not exist"
Sep 16 17:29:01.391008 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.390959    2559 scope.go:117] "RemoveContainer" containerID="31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"
Sep 16 17:29:01.391161 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.391147    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10"} err="failed to get container status \"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10\": rpc error: code = NotFound desc = could not find container \"31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10\": container with ID starting with 31c2564864dd36819bdb27ca6d7d52b1844a1f68547f56c4f429827862014b10 not found: ID does not exist"
Sep 16 17:29:01.391206 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.391161    2559 scope.go:117] "RemoveContainer" containerID="082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"
Sep 16 17:29:01.391354 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.391339    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762"} err="failed to get container status \"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762\": rpc error: code = NotFound desc = could not find container \"082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762\": container with ID starting with 082aff35e566f81d39ea0ba23e8c819eb47cd4ba86b4687cfe6db4a4c896d762 not found: ID does not exist"
Sep 16 17:29:01.401173 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401141    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:29:01.401478 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401463    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="prom-label-proxy"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401480    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="prom-label-proxy"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401490    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="config-reloader"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401496    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="config-reloader"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401504    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerName="registry"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401509    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerName="registry"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401518    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-web"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401523    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-web"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401532    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="init-config-reloader"
Sep 16 17:29:01.401540 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401538    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="init-config-reloader"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401545    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401552    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401574    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="alertmanager"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401579    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="alertmanager"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401586    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-metric"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401591    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-metric"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401646    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-metric"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401655    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy-web"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401661    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9c35d00-3e65-4732-9333-0e2c882d4cde" containerName="registry"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401668    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="prom-label-proxy"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401676    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="kube-rbac-proxy"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401683    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="config-reloader"
Sep 16 17:29:01.401843 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.401688    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3aed0a-6861-4498-852d-76111b24e185" containerName="alertmanager"
Sep 16 17:29:01.406870 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.406847    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.409874 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.409853    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 17:29:01.410314 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410289    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 17:29:01.410410 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410314    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pr7dv\""
Sep 16 17:29:01.410410 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410325    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 17:29:01.410410 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410393    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 17:29:01.410634 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410615    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 17:29:01.410724 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410706    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 17:29:01.410787 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410737    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 17:29:01.411000 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.410982    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 17:29:01.417953 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.417927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 17:29:01.427534 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.427502    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:29:01.502280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wqpq\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-kube-api-access-9wqpq\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502524    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502867 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.502867 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.502692    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.603801 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.603764    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604008 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.603810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604008 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.603929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604134 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604134 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604134 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604197    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604279 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604613 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604613 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wqpq\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-kube-api-access-9wqpq\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.604914 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.605020 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.604910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.605236 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.605211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607064 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607036    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607064 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607269 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607599 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-out\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607713 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607664    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607713 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607667    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.607940 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.607920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.608237 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.608215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-web-config\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.608537 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.608520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ce2394d0-4784-4a22-8477-8421ec5b98fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.624516 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.624491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wqpq\" (UniqueName: \"kubernetes.io/projected/ce2394d0-4784-4a22-8477-8421ec5b98fb-kube-api-access-9wqpq\") pod \"alertmanager-main-0\" (UID: \"ce2394d0-4784-4a22-8477-8421ec5b98fb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:01.718262 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:01.718215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 17:29:02.001498 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:02.001412    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 17:29:02.004876 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:29:02.004849    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2394d0_4784_4a22_8477_8421ec5b98fb.slice/crio-5ab709bc8c3ca285cf9c64f73c44b8617399f119ab89c18086a7b6157824c743 WatchSource:0}: Error finding container 5ab709bc8c3ca285cf9c64f73c44b8617399f119ab89c18086a7b6157824c743: Status 404 returned error can't find the container with id 5ab709bc8c3ca285cf9c64f73c44b8617399f119ab89c18086a7b6157824c743
Sep 16 17:29:02.337511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:02.337474    2559 generic.go:356] "Generic (PLEG): container finished" podID="ce2394d0-4784-4a22-8477-8421ec5b98fb" containerID="773e386e168036066748c12ee5e31f858b8b096fe4515789ff9ec337615a7d58" exitCode=0
Sep 16 17:29:02.337722 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:02.337535    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerDied","Data":"773e386e168036066748c12ee5e31f858b8b096fe4515789ff9ec337615a7d58"}
Sep 16 17:29:02.337722 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:02.337651    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"5ab709bc8c3ca285cf9c64f73c44b8617399f119ab89c18086a7b6157824c743"}
Sep 16 17:29:03.294726 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.294693    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc3aed0a-6861-4498-852d-76111b24e185" path="/var/lib/kubelet/pods/bc3aed0a-6861-4498-852d-76111b24e185/volumes"
Sep 16 17:29:03.345269 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"7d4e8559727c7583713b70f79e68df58d9b96910b2dbb6f20231b8b3d8ffb996"}
Sep 16 17:29:03.345411 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"63d2f899e59d592b65ddf3f2eb2a913635f821282731cc088912fa7b52020dc4"}
Sep 16 17:29:03.345411 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"ba43f67a5ba0603b0b893ff225527d605c6945664b120112c436ac50f1ca2dda"}
Sep 16 17:29:03.345411 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"8c9557ea89888e246ddadf7464f2202ae39b435d5c5781bf88c5352553d60f8a"}
Sep 16 17:29:03.345539 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"97f148f021dc7bca3950bcd1ec487603d43db11963fae1b5a856513a2bb5d0ae"}
Sep 16 17:29:03.345539 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.345469    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ce2394d0-4784-4a22-8477-8421ec5b98fb","Type":"ContainerStarted","Data":"4233d012d86a7145199e6f92b9b1f19c43a1bad39bb4337f72b879f2aa157599"}
Sep 16 17:29:03.375704 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:03.375648    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.359408522 podStartE2EDuration="2.375622654s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.214132ms" isStatefulPod=false podCreationTimestamp="2026-09-16 17:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:29:03.373094343 +0000 UTC m=+266.602876033" watchObservedRunningTime="2026-09-16 17:29:03.375622654 +0000 UTC m=+266.605404345"
Sep 16 17:29:04.017855 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.017815    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:04.018221 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018193    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="prometheus" containerID="cri-o://bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174" gracePeriod=600
Sep 16 17:29:04.018295 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018246    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-thanos" containerID="cri-o://49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20" gracePeriod=600
Sep 16 17:29:04.018354 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018284    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="thanos-sidecar" containerID="cri-o://2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb" gracePeriod=600
Sep 16 17:29:04.018354 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018315    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="config-reloader" containerID="cri-o://ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd" gracePeriod=600
Sep 16 17:29:04.018468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018343    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy" containerID="cri-o://f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37" gracePeriod=600
Sep 16 17:29:04.018468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.018294    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-web" containerID="cri-o://e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7" gracePeriod=600
Sep 16 17:29:04.283591 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.283538    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355681    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355719    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355731    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355743    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355753    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.355763    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2888896-d825-4511-8541-27adaa7f4306" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174" exitCode=0
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357095    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.357651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357637    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357757    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357791    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357841    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"}
Sep 16 17:29:04.358361 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.357922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2888896-d825-4511-8541-27adaa7f4306","Type":"ContainerDied","Data":"d02afd79e52ca31821189b47c824b20d0e412b31dde8dbeac54e256504b69821"}
Sep 16 17:29:04.369249 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.369225    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.378139 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.378118    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.385814 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.385785    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.393772 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.393747    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.401355 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.401335    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.409720 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.409701    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.417177 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.417156    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.417504 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.417483    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.417549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.417517    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.417549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.417535    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.417856 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.417835    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.417913 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.417862    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.417913 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.417878    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.418140 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.418126    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.418177 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418143    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.418177 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418156    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.418356 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.418341    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.418399 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418360    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.418399 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418374    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.418649 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.418625    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.418732 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418653    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.418732 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418670    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.418927 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.418912    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.418977 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418933    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.418977 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.418945    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.419149 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:04.419133    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.419192 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419152    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.419192 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419164    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.419382 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419366    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.419421 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419382    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.419603 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419585    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.419644 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419603    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.419813 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419797    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.419856 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.419812    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.420042 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420026    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.420082 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420042    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.420265 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420251    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.420265 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420265    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.420461 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420445    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.420503 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420462    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.420658 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420642    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.420705 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420658    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.420882 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420867    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.420882 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.420881    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.421062 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421047    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.421101 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421062    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.421280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421259    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.421280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421280    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.421523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421503    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.421588 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421522    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.421812 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421793    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.421857 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.421812    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.422050 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422032    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.422090 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422051    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.422273 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422258    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.422320 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422274    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.422487 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422472    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.422528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422487    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.422723 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422705    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.422773 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422723    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.422965 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422948    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.423009 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.422966    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.423180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423164    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.423221 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423180    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.423400 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423384    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.423449 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423401    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.423622 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423606    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.423672 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423622    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.423848 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423831    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.423888 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.423849    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.424065 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424048    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.424111 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424065    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.424281 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424264    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.424326 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424281    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.424485 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424470    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.424521 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424485    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.424736 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424719    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.424791 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424737    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.424952 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424935    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.424992 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.424952    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.425200 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425183    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.425200 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425199    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.425412 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425394    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.425450 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425412    2559 scope.go:117] "RemoveContainer" containerID="49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"
Sep 16 17:29:04.425635 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425620    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20"} err="failed to get container status \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": rpc error: code = NotFound desc = could not find container \"49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20\": container with ID starting with 49935fe81aeba7ebb1181acba5f7d876d8a72086b6e074953a7d34a781d54b20 not found: ID does not exist"
Sep 16 17:29:04.425635 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425635    2559 scope.go:117] "RemoveContainer" containerID="f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"
Sep 16 17:29:04.425855 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425839    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37"} err="failed to get container status \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": rpc error: code = NotFound desc = could not find container \"f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37\": container with ID starting with f7690e875aa1ac246e0a1188318d2bd29fc0b3c7a0aad70fd846df23e699fd37 not found: ID does not exist"
Sep 16 17:29:04.425896 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.425855    2559 scope.go:117] "RemoveContainer" containerID="e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"
Sep 16 17:29:04.426068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426053    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7"} err="failed to get container status \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": rpc error: code = NotFound desc = could not find container \"e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7\": container with ID starting with e018c9b017c94909996ab45ab084f8c7e62b08b95ac4c7a709660ed96fc322d7 not found: ID does not exist"
Sep 16 17:29:04.426110 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426068    2559 scope.go:117] "RemoveContainer" containerID="2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"
Sep 16 17:29:04.426257 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426243    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb"} err="failed to get container status \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": rpc error: code = NotFound desc = could not find container \"2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb\": container with ID starting with 2467cc98e2ce9b1fad9687b39b3db095c8541821d309a0f03188f6aa540d81fb not found: ID does not exist"
Sep 16 17:29:04.426294 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426257    2559 scope.go:117] "RemoveContainer" containerID="ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"
Sep 16 17:29:04.426468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426452    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd"} err="failed to get container status \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": rpc error: code = NotFound desc = could not find container \"ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd\": container with ID starting with ed8333bd8dbb34cb78d8ece03a61ccc7e54aeb9e821185ad3c38b2f1feef92fd not found: ID does not exist"
Sep 16 17:29:04.426468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426467    2559 scope.go:117] "RemoveContainer" containerID="bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"
Sep 16 17:29:04.426707 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426692    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174"} err="failed to get container status \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": rpc error: code = NotFound desc = could not find container \"bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174\": container with ID starting with bbe848c4ce5fc3126954bb891fe62b53670fb4b04ca4fc9beeff991605c1b174 not found: ID does not exist"
Sep 16 17:29:04.426751 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426707    2559 scope.go:117] "RemoveContainer" containerID="d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"
Sep 16 17:29:04.426910 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.426896    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69"} err="failed to get container status \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": rpc error: code = NotFound desc = could not find container \"d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69\": container with ID starting with d445cf62224efde3e738fd1cf85f790e69b91e45c905441b3a964f949a0c8a69 not found: ID does not exist"
Sep 16 17:29:04.437062 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437039    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437172 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437071    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437172 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437119    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437172 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437151    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437193    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437219    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437262    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437296    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437457    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437478    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.437511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437494    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437678 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437520    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.437754 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437590    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437818 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437780    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437818 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437810    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437915 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437837    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437915 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437862    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.437915 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437887    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55phv\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.438068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437939    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.438068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437967    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.438068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.437991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.438068 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.438024    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls\") pod \"e2888896-d825-4511-8541-27adaa7f4306\" (UID: \"e2888896-d825-4511-8541-27adaa7f4306\") "
Sep 16 17:29:04.438313 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.438286    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.438373 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.438324    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.438373 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.438339    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.439140 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.439018    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:29:04.439896 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.439753    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.440207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.439896    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:29:04.440207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.440080    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.440511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.440377    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config" (OuterVolumeSpecName: "config") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.440511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.440379    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.440998 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.440953    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:29:04.441153 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.441024    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.442121 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.442098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv" (OuterVolumeSpecName: "kube-api-access-55phv") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "kube-api-access-55phv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:29:04.442393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.442352    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.443280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.443248    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.443604 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.443547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out" (OuterVolumeSpecName: "config-out") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:29:04.443984 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.443958    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.444180 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.444157    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.444365 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.444337    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.445280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.445257    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.454175 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.454146    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config" (OuterVolumeSpecName: "web-config") pod "e2888896-d825-4511-8541-27adaa7f4306" (UID: "e2888896-d825-4511-8541-27adaa7f4306"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540152    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-grpc-tls\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540186    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540197    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540207    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540217    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540226    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-db\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540236    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-metrics-client-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540244    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540260 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540253    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540276    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540298    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-kube-rbac-proxy\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540310    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55phv\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-kube-api-access-55phv\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540322    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-web-config\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540335    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2888896-d825-4511-8541-27adaa7f4306-configmap-metrics-client-ca\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540346    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2888896-d825-4511-8541-27adaa7f4306-config-out\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540358    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540370    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2888896-d825-4511-8541-27adaa7f4306-config\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.540703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.540381    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2888896-d825-4511-8541-27adaa7f4306-tls-assets\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:29:04.686654 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.686617    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:04.693884 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.693859    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:04.724659 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.724630    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:04.725069 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725051    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="thanos-sidecar"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725070    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="thanos-sidecar"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725079    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-web"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725085    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-web"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725094    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="config-reloader"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725099    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="config-reloader"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725109    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="prometheus"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725114    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="prometheus"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725121    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-thanos"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-thanos"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725138    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy"
Sep 16 17:29:04.725145 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725143    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725151    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="init-config-reloader"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725156    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="init-config-reloader"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725205    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="prometheus"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725212    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="config-reloader"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725219    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-thanos"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725226    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="thanos-sidecar"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725233    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy"
Sep 16 17:29:04.725448 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.725238    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2888896-d825-4511-8541-27adaa7f4306" containerName="kube-rbac-proxy-web"
Sep 16 17:29:04.730781 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.730758    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.737984 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.737725    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 17:29:04.737984 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.737760    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 17:29:04.737984 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.737805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 17:29:04.738168 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738060    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 17:29:04.738168 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738114    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 17:29:04.738266 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738240    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 17:29:04.738314 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738302    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 17:29:04.738953 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738916    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 17:29:04.738953 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.738934    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 17:29:04.739262 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.739245    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3lsd4ju8kvk6k\""
Sep 16 17:29:04.740373 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.740355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 17:29:04.740476 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.740463    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 17:29:04.741243 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.741228    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zhmxg\""
Sep 16 17:29:04.741363 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.741347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 17:29:04.742098 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.742080    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 17:29:04.745721 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.745703    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 17:29:04.749359 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.749339    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:04.841708 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841674    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841720    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-config-out\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841824    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841844    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.841898 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.841987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842151    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842182 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842180    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cv8b\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-kube-api-access-4cv8b\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-web-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.842457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.842252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.942712 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.942680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.942883 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.942729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.942883 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.942768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.942883 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.942790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943016 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.942989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943062 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943109 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cv8b\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-kube-api-access-4cv8b\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943166 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-web-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943166 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943414 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-config-out\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943761    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943797    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.943873 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.944469 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.944664 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.944639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.945471 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.945445    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946495 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.945800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946495 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-web-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946495 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946495 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946495 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.943518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.946807 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.947297 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-config\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.947297 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.946941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.947297 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.947237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b287a8-ec65-4317-bccb-715e8c03f3a0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.947502 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.947369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.948173 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.948147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.948603 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.948583    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01b287a8-ec65-4317-bccb-715e8c03f3a0-config-out\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.948677 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.948616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.948989 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.948970    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.949049 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.949022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/01b287a8-ec65-4317-bccb-715e8c03f3a0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:04.951696 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:04.951671    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cv8b\" (UniqueName: \"kubernetes.io/projected/01b287a8-ec65-4317-bccb-715e8c03f3a0-kube-api-access-4cv8b\") pod \"prometheus-k8s-0\" (UID: \"01b287a8-ec65-4317-bccb-715e8c03f3a0\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:05.041525 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:05.041489    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:05.294742 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:05.294702    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2888896-d825-4511-8541-27adaa7f4306" path="/var/lib/kubelet/pods/e2888896-d825-4511-8541-27adaa7f4306/volumes"
Sep 16 17:29:05.324783 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:05.324757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 17:29:05.326637 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:29:05.326600    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b287a8_ec65_4317_bccb_715e8c03f3a0.slice/crio-272218e0402e97ce94704b95bab8de1153500ce57967b97341bddfb198a57663 WatchSource:0}: Error finding container 272218e0402e97ce94704b95bab8de1153500ce57967b97341bddfb198a57663: Status 404 returned error can't find the container with id 272218e0402e97ce94704b95bab8de1153500ce57967b97341bddfb198a57663
Sep 16 17:29:05.362618 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:05.362582    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"272218e0402e97ce94704b95bab8de1153500ce57967b97341bddfb198a57663"}
Sep 16 17:29:06.370511 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:06.370475    2559 generic.go:356] "Generic (PLEG): container finished" podID="01b287a8-ec65-4317-bccb-715e8c03f3a0" containerID="0d534e708bab678faae4a884975bafd3e65496f061dcd29f64454dc19ea66c63" exitCode=0
Sep 16 17:29:06.370881 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:06.370545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerDied","Data":"0d534e708bab678faae4a884975bafd3e65496f061dcd29f64454dc19ea66c63"}
Sep 16 17:29:07.378928 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.378886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"8479cde3c2ea2ea7dfa461b883ac943c59301ba82ae69fd53ec9d92578537728"}
Sep 16 17:29:07.379336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.378973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"0582fbaea2f9449bba0f6717c9f945b531e3554518b3f9a08454bd89e8b171a3"}
Sep 16 17:29:07.379336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.379016    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"78601d919f6668972725f74e0d6a8e818bb770b9f56ace39b9cd3f99b97112fb"}
Sep 16 17:29:07.379336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.379057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"fa3aa44d4605aede78896bbf976e487a8c718ea75e0ce39c2d540ea7499bd009"}
Sep 16 17:29:07.379336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.379090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"306657a9b029e14c24fca21324e7404174aeeef322ddb610a72a94c1b0dfdab1"}
Sep 16 17:29:07.379336 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.379124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"01b287a8-ec65-4317-bccb-715e8c03f3a0","Type":"ContainerStarted","Data":"ef0f27a74b7a0473778cf3ae472d5269d1ebf643eb90c992fffb0ac8a8322512"}
Sep 16 17:29:07.415692 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:07.415631    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.397262109 podStartE2EDuration="3.415613208s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.351099ms" isStatefulPod=false podCreationTimestamp="2026-09-16 17:29:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:29:07.414081566 +0000 UTC m=+270.643863281" watchObservedRunningTime="2026-09-16 17:29:07.415613208 +0000 UTC m=+270.645394919"
Sep 16 17:29:10.042061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:10.042019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:29:16.823338 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:29:16.823292    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5" podUID="1b5a31bf-d965-41a6-8e2d-57a5582241a9"
Sep 16 17:29:17.423616 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:17.423581    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:29:20.588691 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:20.588638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:29:20.591292 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:20.591259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b5a31bf-d965-41a6-8e2d-57a5582241a9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jl5w5\" (UID: \"1b5a31bf-d965-41a6-8e2d-57a5582241a9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:29:20.727119 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:20.727087    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xtd7w\""
Sep 16 17:29:20.735269 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:20.735243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"
Sep 16 17:29:21.013795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:21.013763    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5"]
Sep 16 17:29:21.016381 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:29:21.016348    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5a31bf_d965_41a6_8e2d_57a5582241a9.slice/crio-d0a4e3641d17d43cf447d4c13da0505f2fcb4b7c9515e3536a2312d96fe59771 WatchSource:0}: Error finding container d0a4e3641d17d43cf447d4c13da0505f2fcb4b7c9515e3536a2312d96fe59771: Status 404 returned error can't find the container with id d0a4e3641d17d43cf447d4c13da0505f2fcb4b7c9515e3536a2312d96fe59771
Sep 16 17:29:21.441468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:21.441427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5" event={"ID":"1b5a31bf-d965-41a6-8e2d-57a5582241a9","Type":"ContainerStarted","Data":"d0a4e3641d17d43cf447d4c13da0505f2fcb4b7c9515e3536a2312d96fe59771"}
Sep 16 17:29:22.447476 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:22.447382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5" event={"ID":"1b5a31bf-d965-41a6-8e2d-57a5582241a9","Type":"ContainerStarted","Data":"68e36329f68a3184e70603e79e51c4e210c3c86178790a3e9b2281c959d8eaf7"}
Sep 16 17:29:22.467160 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:22.467104    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jl5w5" podStartSLOduration=284.287776795 podStartE2EDuration="4m45.467087973s" totalImagesPullingTime="1.179311178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:29:22.466195082 +0000 UTC m=+285.695976772" watchObservedRunningTime="2026-09-16 17:29:22.467087973 +0000 UTC m=+285.696869663"
Sep 16 17:29:37.193938 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:37.193906    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:29:37.198622 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:37.198593    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:29:37.207323 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:29:37.207297    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 17:30:05.041972 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:30:05.041930    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:30:05.057827 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:30:05.057801    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:30:05.642973 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:30:05.642939    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 17:34:27.316444 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.316408    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mgwf"]
Sep 16 17:34:27.319739 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.319717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.328100 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.328076    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 17:34:27.328205 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.328084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 17:34:27.353571 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.353528    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mgwf"]
Sep 16 17:34:27.362479 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.362446    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.362691 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.362500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbs8\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-kube-api-access-bsbs8\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.464041 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.464004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbs8\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-kube-api-access-bsbs8\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.464229 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.464100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.489314 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.489272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbs8\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-kube-api-access-bsbs8\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.490977 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.490956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce06437-f8c9-4092-b680-5e69ea435233-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mgwf\" (UID: \"2ce06437-f8c9-4092-b680-5e69ea435233\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.648613 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.648485    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:27.945728 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.945624    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mgwf"]
Sep 16 17:34:27.956500 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:27.956480    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:34:28.729692 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:28.729652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf" event={"ID":"2ce06437-f8c9-4092-b680-5e69ea435233","Type":"ContainerStarted","Data":"c09fae57b52a92c0da88fe703e85f2ae0cc5c188413b1147e10d5cd7981055e6"}
Sep 16 17:34:31.745221 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:31.745186    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf" event={"ID":"2ce06437-f8c9-4092-b680-5e69ea435233","Type":"ContainerStarted","Data":"dac7930c90643e75952102f4a352e400abc231e081a8d18341f1e77b776bc91b"}
Sep 16 17:34:31.745704 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:31.745260    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:34:31.782717 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:31.782655    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf" podStartSLOduration=1.612729666 podStartE2EDuration="4.782638292s" totalImagesPullingTime="3.169908626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:34:31.781398615 +0000 UTC m=+595.011180307" watchObservedRunningTime="2026-09-16 17:34:31.782638292 +0000 UTC m=+595.012419982"
Sep 16 17:34:37.226344 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:37.226309    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:34:37.227970 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:37.227946    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:34:37.751921 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:34:37.751888    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mgwf"
Sep 16 17:35:12.519656 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.519600    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"]
Sep 16 17:35:12.523974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.523947    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.527061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.527034    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 17:35:12.527231 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.527079    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 17:35:12.528023 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.528002    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 17:35:12.528265 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.528248    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-whtnm\""
Sep 16 17:35:12.528350 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.528247    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:35:12.528479 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.528465    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 17:35:12.544295 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.544258    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"]
Sep 16 17:35:12.587670 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.587624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-metrics-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.587868 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.587677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.587868 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.587792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swrbn\" (UniqueName: \"kubernetes.io/projected/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-kube-api-access-swrbn\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.587868 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.587841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-manager-config\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.689289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.689241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-metrics-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.689289 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.689285    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.689514 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.689418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swrbn\" (UniqueName: \"kubernetes.io/projected/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-kube-api-access-swrbn\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.689514 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.689480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-manager-config\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.690155 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.690130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-manager-config\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.692092 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.692068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.692092 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.692081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-metrics-cert\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.702178 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.702150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swrbn\" (UniqueName: \"kubernetes.io/projected/97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687-kube-api-access-swrbn\") pod \"lws-controller-manager-86b645b5b5-bgjcv\" (UID: \"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687\") " pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:12.834331 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:12.834281    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:13.134759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:13.134679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"]
Sep 16 17:35:13.137450 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:35:13.137422    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bcbee0_ad5a_4cd8_b894_bdeb1fe3c687.slice/crio-e0c53c46dd2d631f89688e43562a063116e0f1b4c5bf697cd3deb41b0227c755 WatchSource:0}: Error finding container e0c53c46dd2d631f89688e43562a063116e0f1b4c5bf697cd3deb41b0227c755: Status 404 returned error can't find the container with id e0c53c46dd2d631f89688e43562a063116e0f1b4c5bf697cd3deb41b0227c755
Sep 16 17:35:13.922420 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:13.922381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv" event={"ID":"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687","Type":"ContainerStarted","Data":"e0c53c46dd2d631f89688e43562a063116e0f1b4c5bf697cd3deb41b0227c755"}
Sep 16 17:35:15.933291 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:15.933203    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv" event={"ID":"97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687","Type":"ContainerStarted","Data":"be23d0bfe242546abe40553bab4491f3e78122d0611bdce134c07fdf02acdbb6"}
Sep 16 17:35:15.933665 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:15.933329    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:15.956322 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:15.956256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv" podStartSLOduration=1.445398003 podStartE2EDuration="3.956237377s" totalImagesPullingTime="2.510839374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:35:15.954017714 +0000 UTC m=+639.183799407" watchObservedRunningTime="2026-09-16 17:35:15.956237377 +0000 UTC m=+639.186019067"
Sep 16 17:35:26.939700 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:26.939665    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86b645b5b5-bgjcv"
Sep 16 17:35:41.531061 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.531023    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"]
Sep 16 17:35:41.535763 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.535734    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.539280 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.539131    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 17:35:41.539521 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.539498    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6fkj4\""
Sep 16 17:35:41.539851 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.539825    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 17:35:41.558314 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.558278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"]
Sep 16 17:35:41.656499 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.656463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238a3d8c-dea6-4b4e-b89e-3e27447d9901-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.656706 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.656510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqfgs\" (UniqueName: \"kubernetes.io/projected/238a3d8c-dea6-4b4e-b89e-3e27447d9901-kube-api-access-bqfgs\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.757392 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.757349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238a3d8c-dea6-4b4e-b89e-3e27447d9901-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.757392 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.757400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqfgs\" (UniqueName: \"kubernetes.io/projected/238a3d8c-dea6-4b4e-b89e-3e27447d9901-kube-api-access-bqfgs\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.757858 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.757835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/238a3d8c-dea6-4b4e-b89e-3e27447d9901-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.766960 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.766936    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqfgs\" (UniqueName: \"kubernetes.io/projected/238a3d8c-dea6-4b4e-b89e-3e27447d9901-kube-api-access-bqfgs\") pod \"kuadrant-operator-controller-manager-6d4cb79c-8vm4x\" (UID: \"238a3d8c-dea6-4b4e-b89e-3e27447d9901\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:41.848865 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:41.848822    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:42.131390 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:42.131309    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"]
Sep 16 17:35:42.135258 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:35:42.135234    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238a3d8c_dea6_4b4e_b89e_3e27447d9901.slice/crio-14b8c22a439151c7dbe47925a574ec816e5039e6604c6bf4ee16590ac0b36ef0 WatchSource:0}: Error finding container 14b8c22a439151c7dbe47925a574ec816e5039e6604c6bf4ee16590ac0b36ef0: Status 404 returned error can't find the container with id 14b8c22a439151c7dbe47925a574ec816e5039e6604c6bf4ee16590ac0b36ef0
Sep 16 17:35:43.048694 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:43.048660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x" event={"ID":"238a3d8c-dea6-4b4e-b89e-3e27447d9901","Type":"ContainerStarted","Data":"14b8c22a439151c7dbe47925a574ec816e5039e6604c6bf4ee16590ac0b36ef0"}
Sep 16 17:35:48.077222 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:48.077116    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x" event={"ID":"238a3d8c-dea6-4b4e-b89e-3e27447d9901","Type":"ContainerStarted","Data":"ca204d849962cc431cf3e643ad860db05bad18309aedf22434e8d3a4ffff8b59"}
Sep 16 17:35:48.077222 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:48.077215    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:35:48.115398 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:48.115289    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x" podStartSLOduration=1.429415399 podStartE2EDuration="7.115266759s" totalImagesPullingTime="5.68585136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:35:48.114085246 +0000 UTC m=+671.343866932" watchObservedRunningTime="2026-09-16 17:35:48.115266759 +0000 UTC m=+671.345048461"
Sep 16 17:35:59.084609 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:35:59.084572    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-8vm4x"
Sep 16 17:36:34.524039 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.523953    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:34.526389 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.526367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.529315 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.529291    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 17:36:34.529426 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.529291    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2xswh\""
Sep 16 17:36:34.537342 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.537314    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:34.625384 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.625339    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:34.636926 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.636882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcqnc\" (UniqueName: \"kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.637095 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.637003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.737702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.737666    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcqnc\" (UniqueName: \"kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.737894 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.737762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.738438 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.738415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.752334 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.752298    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcqnc\" (UniqueName: \"kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc\") pod \"limitador-limitador-8588749977-tn9wz\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") " pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:34.838406 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:34.838367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:35.124859 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.124827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:35.125685 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:36:35.125660    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2faaba02_f912_46e0_845c_7618a42fdb1c.slice/crio-e64c50671d4f2db04455d86d53017ea810c9e0fcbd094331deba35dac04d080d WatchSource:0}: Error finding container e64c50671d4f2db04455d86d53017ea810c9e0fcbd094331deba35dac04d080d: Status 404 returned error can't find the container with id e64c50671d4f2db04455d86d53017ea810c9e0fcbd094331deba35dac04d080d
Sep 16 17:36:35.300228 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.300194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" event={"ID":"2faaba02-f912-46e0-845c-7618a42fdb1c","Type":"ContainerStarted","Data":"e64c50671d4f2db04455d86d53017ea810c9e0fcbd094331deba35dac04d080d"}
Sep 16 17:36:35.567535 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.567500    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:36:35.570265 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.570243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:35.573323 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.573298    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rbkn4\""
Sep 16 17:36:35.586219 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.586189    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:36:35.646950 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.646913    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fvtq\" (UniqueName: \"kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq\") pod \"authorino-59c974bd48-52v88\" (UID: \"21d86166-186b-46f0-b4f7-65032737c806\") " pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:35.748318 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.748276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fvtq\" (UniqueName: \"kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq\") pod \"authorino-59c974bd48-52v88\" (UID: \"21d86166-186b-46f0-b4f7-65032737c806\") " pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:35.767817 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.767783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fvtq\" (UniqueName: \"kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq\") pod \"authorino-59c974bd48-52v88\" (UID: \"21d86166-186b-46f0-b4f7-65032737c806\") " pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:35.880362 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:35.880268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:36.152066 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:36.152033    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:36:36.157814 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:36:36.157775    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21d86166_186b_46f0_b4f7_65032737c806.slice/crio-486e4a96a72ed4707a461b80866b9bdb6a9c21d6973dcdb0cbe28e5e233a36b5 WatchSource:0}: Error finding container 486e4a96a72ed4707a461b80866b9bdb6a9c21d6973dcdb0cbe28e5e233a36b5: Status 404 returned error can't find the container with id 486e4a96a72ed4707a461b80866b9bdb6a9c21d6973dcdb0cbe28e5e233a36b5
Sep 16 17:36:36.307009 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:36.306966    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-52v88" event={"ID":"21d86166-186b-46f0-b4f7-65032737c806","Type":"ContainerStarted","Data":"486e4a96a72ed4707a461b80866b9bdb6a9c21d6973dcdb0cbe28e5e233a36b5"}
Sep 16 17:36:37.315212 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:37.315171    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" event={"ID":"2faaba02-f912-46e0-845c-7618a42fdb1c","Type":"ContainerStarted","Data":"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"}
Sep 16 17:36:37.315651 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:37.315283    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:37.343800 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:37.343745    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" podStartSLOduration=2.071543206 podStartE2EDuration="3.343721016s" totalImagesPullingTime="1.27217781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:36:37.339619815 +0000 UTC m=+720.569401504" watchObservedRunningTime="2026-09-16 17:36:37.343721016 +0000 UTC m=+720.573502709"
Sep 16 17:36:39.327601 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:39.327548    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-52v88" event={"ID":"21d86166-186b-46f0-b4f7-65032737c806","Type":"ContainerStarted","Data":"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"}
Sep 16 17:36:39.344590 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:39.344521    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-52v88" podStartSLOduration=2.125360946 podStartE2EDuration="4.344502921s" totalImagesPullingTime="2.219141975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:36:39.342882531 +0000 UTC m=+722.572664224" watchObservedRunningTime="2026-09-16 17:36:39.344502921 +0000 UTC m=+722.574284612"
Sep 16 17:36:48.109981 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.109942    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:48.110458 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.110134    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" podUID="2faaba02-f912-46e0-845c-7618a42fdb1c" containerName="limitador" containerID="cri-o://4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936" gracePeriod=30
Sep 16 17:36:48.110912 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.110890    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:48.659208 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.659180    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:48.766983 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.766892    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcqnc\" (UniqueName: \"kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc\") pod \"2faaba02-f912-46e0-845c-7618a42fdb1c\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") "
Sep 16 17:36:48.766983 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.766950    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file\") pod \"2faaba02-f912-46e0-845c-7618a42fdb1c\" (UID: \"2faaba02-f912-46e0-845c-7618a42fdb1c\") "
Sep 16 17:36:48.767364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.767343    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file" (OuterVolumeSpecName: "config-file") pod "2faaba02-f912-46e0-845c-7618a42fdb1c" (UID: "2faaba02-f912-46e0-845c-7618a42fdb1c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:36:48.769271 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.769245    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc" (OuterVolumeSpecName: "kube-api-access-vcqnc") pod "2faaba02-f912-46e0-845c-7618a42fdb1c" (UID: "2faaba02-f912-46e0-845c-7618a42fdb1c"). InnerVolumeSpecName "kube-api-access-vcqnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:36:48.867925 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.867886    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcqnc\" (UniqueName: \"kubernetes.io/projected/2faaba02-f912-46e0-845c-7618a42fdb1c-kube-api-access-vcqnc\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:36:48.867925 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:48.867920    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2faaba02-f912-46e0-845c-7618a42fdb1c-config-file\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:36:49.373666 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.373628    2559 generic.go:356] "Generic (PLEG): container finished" podID="2faaba02-f912-46e0-845c-7618a42fdb1c" containerID="4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936" exitCode=0
Sep 16 17:36:49.374154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.373677    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz"
Sep 16 17:36:49.374154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.373713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" event={"ID":"2faaba02-f912-46e0-845c-7618a42fdb1c","Type":"ContainerDied","Data":"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"}
Sep 16 17:36:49.374154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.373778    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tn9wz" event={"ID":"2faaba02-f912-46e0-845c-7618a42fdb1c","Type":"ContainerDied","Data":"e64c50671d4f2db04455d86d53017ea810c9e0fcbd094331deba35dac04d080d"}
Sep 16 17:36:49.374154 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.373813    2559 scope.go:117] "RemoveContainer" containerID="4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"
Sep 16 17:36:49.382543 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.382507    2559 scope.go:117] "RemoveContainer" containerID="4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"
Sep 16 17:36:49.382853 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:36:49.382834    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936\": container with ID starting with 4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936 not found: ID does not exist" containerID="4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"
Sep 16 17:36:49.382905 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.382863    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936"} err="failed to get container status \"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936\": rpc error: code = NotFound desc = could not find container \"4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936\": container with ID starting with 4840e7c8e35c5b6a65d77c228b445b9118aed7b836b12ed65ba6db2b1ee48936 not found: ID does not exist"
Sep 16 17:36:49.391874 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.391844    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:49.396192 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:49.396167    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tn9wz"]
Sep 16 17:36:51.295582 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:51.295529    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2faaba02-f912-46e0-845c-7618a42fdb1c" path="/var/lib/kubelet/pods/2faaba02-f912-46e0-845c-7618a42fdb1c/volumes"
Sep 16 17:36:58.845698 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:58.845656    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:36:58.846155 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:58.845838    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-52v88" podUID="21d86166-186b-46f0-b4f7-65032737c806" containerName="authorino" containerID="cri-o://09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c" gracePeriod=30
Sep 16 17:36:59.096067 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.096002    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:59.150209 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.150171    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fvtq\" (UniqueName: \"kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq\") pod \"21d86166-186b-46f0-b4f7-65032737c806\" (UID: \"21d86166-186b-46f0-b4f7-65032737c806\") "
Sep 16 17:36:59.152339 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.152297    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq" (OuterVolumeSpecName: "kube-api-access-8fvtq") pod "21d86166-186b-46f0-b4f7-65032737c806" (UID: "21d86166-186b-46f0-b4f7-65032737c806"). InnerVolumeSpecName "kube-api-access-8fvtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:36:59.250797 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.250765    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fvtq\" (UniqueName: \"kubernetes.io/projected/21d86166-186b-46f0-b4f7-65032737c806-kube-api-access-8fvtq\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:36:59.430833 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.430740    2559 generic.go:356] "Generic (PLEG): container finished" podID="21d86166-186b-46f0-b4f7-65032737c806" containerID="09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c" exitCode=0
Sep 16 17:36:59.430833 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.430782    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-52v88"
Sep 16 17:36:59.430833 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.430802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-52v88" event={"ID":"21d86166-186b-46f0-b4f7-65032737c806","Type":"ContainerDied","Data":"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"}
Sep 16 17:36:59.431050 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.430841    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-52v88" event={"ID":"21d86166-186b-46f0-b4f7-65032737c806","Type":"ContainerDied","Data":"486e4a96a72ed4707a461b80866b9bdb6a9c21d6973dcdb0cbe28e5e233a36b5"}
Sep 16 17:36:59.431050 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.430858    2559 scope.go:117] "RemoveContainer" containerID="09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"
Sep 16 17:36:59.439429 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.439410    2559 scope.go:117] "RemoveContainer" containerID="09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"
Sep 16 17:36:59.439752 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:36:59.439730    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c\": container with ID starting with 09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c not found: ID does not exist" containerID="09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"
Sep 16 17:36:59.439821 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.439760    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c"} err="failed to get container status \"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c\": rpc error: code = NotFound desc = could not find container \"09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c\": container with ID starting with 09e76ac5da4d1e65c79a77f4ffdeaa58c65980acb9ac55e2e2199aa4a9cdfd2c not found: ID does not exist"
Sep 16 17:36:59.448009 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.447983    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:36:59.453968 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:36:59.453943    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-52v88"]
Sep 16 17:37:01.295241 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:01.295196    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d86166-186b-46f0-b4f7-65032737c806" path="/var/lib/kubelet/pods/21d86166-186b-46f0-b4f7-65032737c806/volumes"
Sep 16 17:37:19.779044 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779012    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"]
Sep 16 17:37:19.779457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779350    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2faaba02-f912-46e0-845c-7618a42fdb1c" containerName="limitador"
Sep 16 17:37:19.779457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779363    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2faaba02-f912-46e0-845c-7618a42fdb1c" containerName="limitador"
Sep 16 17:37:19.779457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779386    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d86166-186b-46f0-b4f7-65032737c806" containerName="authorino"
Sep 16 17:37:19.779457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779391    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d86166-186b-46f0-b4f7-65032737c806" containerName="authorino"
Sep 16 17:37:19.779457 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779455    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2faaba02-f912-46e0-845c-7618a42fdb1c" containerName="limitador"
Sep 16 17:37:19.779639 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.779468    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d86166-186b-46f0-b4f7-65032737c806" containerName="authorino"
Sep 16 17:37:19.781900 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.781880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.784528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.784507    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 17:37:19.784674 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.784623    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 17:37:19.785523 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.785499    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 17:37:19.785640 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.785508    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 17:37:19.785640 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.785613    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 17:37:19.785759 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.785649    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hg6z6\""
Sep 16 17:37:19.790528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.790498    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 17:37:19.803620 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.803598    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"]
Sep 16 17:37:19.833538 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833656 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833656 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b435325-ea04-4231-8d85-a3c6cdfd52fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833749 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833749 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkxc\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-kube-api-access-bqkxc\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833831 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833769    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.833831 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.833789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.934799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.934903 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.934903 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b435325-ea04-4231-8d85-a3c6cdfd52fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.934903 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.935080 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkxc\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-kube-api-access-bqkxc\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.935080 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.935080 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.934983    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.935554 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.935529    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.937312 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.937286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b435325-ea04-4231-8d85-a3c6cdfd52fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.937587 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.937551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.937637 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.937602    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.937715 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.937695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.943701 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.943684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:19.944131 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:19.944109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkxc\" (UniqueName: \"kubernetes.io/projected/2b435325-ea04-4231-8d85-a3c6cdfd52fe-kube-api-access-bqkxc\") pod \"istiod-openshift-gateway-5677ccb49b-k4rtb\" (UID: \"2b435325-ea04-4231-8d85-a3c6cdfd52fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:20.091597 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:20.091547    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:20.366365 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:20.366340    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"]
Sep 16 17:37:20.367882 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:37:20.367860    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b435325_ea04_4231_8d85_a3c6cdfd52fe.slice/crio-bcb0654789548b51b923585ae330c48b12d4dfbc177a9c1913014f9fa6925ac1 WatchSource:0}: Error finding container bcb0654789548b51b923585ae330c48b12d4dfbc177a9c1913014f9fa6925ac1: Status 404 returned error can't find the container with id bcb0654789548b51b923585ae330c48b12d4dfbc177a9c1913014f9fa6925ac1
Sep 16 17:37:20.530364 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:20.530329    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb" event={"ID":"2b435325-ea04-4231-8d85-a3c6cdfd52fe","Type":"ContainerStarted","Data":"bcb0654789548b51b923585ae330c48b12d4dfbc177a9c1913014f9fa6925ac1"}
Sep 16 17:37:22.753122 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:22.753081    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:37:22.753393 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:22.753149    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:37:23.547449 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:23.547419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb" event={"ID":"2b435325-ea04-4231-8d85-a3c6cdfd52fe","Type":"ContainerStarted","Data":"9606d2a8f6763344ac68311d1f4324eadb0a22373da19a9fb8eea2b565be4d82"}
Sep 16 17:37:23.547634 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:23.547586    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:23.549234 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:23.549214    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb"
Sep 16 17:37:23.587832 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:37:23.587784    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k4rtb" podStartSLOduration=2.204630873 podStartE2EDuration="4.587764625s" totalImagesPullingTime="2.383133752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:37:23.584484876 +0000 UTC m=+766.814266567" watchObservedRunningTime="2026-09-16 17:37:23.587764625 +0000 UTC m=+766.817546329"
Sep 16 17:39:37.256304 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:39:37.256223    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:39:37.256746 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:39:37.256445    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:44:37.284636 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:44:37.284602    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:44:37.285699 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:44:37.285680    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:49:37.312582 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:49:37.312538    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:49:37.315201 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:49:37.315174    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:52:08.826469 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.826431    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:52:08.829983 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.829958    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.833236 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.833214    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-vm4mf\""
Sep 16 17:52:08.833373 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.833215    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Sep 16 17:52:08.833544 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.833527    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-2ncbp\""
Sep 16 17:52:08.833652 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.833533    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Sep 16 17:52:08.833905 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.833887    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Sep 16 17:52:08.844744 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.844716    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:52:08.861136 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.861091    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.861305 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.861145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc6r4\" (UniqueName: \"kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.962217 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.962176    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.962413 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.962235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc6r4\" (UniqueName: \"kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.964808 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.964784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:08.972193 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:08.972163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc6r4\" (UniqueName: \"kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:09.140822 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:09.140735    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:09.417988 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:09.417913    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:52:09.420335 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:52:09.420306    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5bb4e1d_af36_4378_bfe2_586aefd598f0.slice/crio-6059a575a08e5367dfbd528e76824e5a5a35d0252ef41874b429cfb987b0000f WatchSource:0}: Error finding container 6059a575a08e5367dfbd528e76824e5a5a35d0252ef41874b429cfb987b0000f: Status 404 returned error can't find the container with id 6059a575a08e5367dfbd528e76824e5a5a35d0252ef41874b429cfb987b0000f
Sep 16 17:52:09.422350 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:09.422331    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:52:09.561047 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:09.561013    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" event={"ID":"d5bb4e1d-af36-4378-bfe2-586aefd598f0","Type":"ContainerStarted","Data":"6059a575a08e5367dfbd528e76824e5a5a35d0252ef41874b429cfb987b0000f"}
Sep 16 17:52:12.578855 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:12.578816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" event={"ID":"d5bb4e1d-af36-4378-bfe2-586aefd598f0","Type":"ContainerStarted","Data":"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"}
Sep 16 17:52:12.579250 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:12.578902    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:52:12.604612 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:12.604505    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podStartSLOduration=2.327557277 podStartE2EDuration="4.60449217s" totalImagesPullingTime="2.276934893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:52:12.603609452 +0000 UTC m=+1655.833391142" watchObservedRunningTime="2026-09-16 17:52:12.60449217 +0000 UTC m=+1655.834273860"
Sep 16 17:52:43.585292 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:43.585256    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:52:43.937726 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:43.937648    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:52:43.939465 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:43.939418    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:52:53.939468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:53.939428    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:52:58.958161 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:52:58.958097    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" containerID="cri-o://9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591" gracePeriod=45
Sep 16 17:53:03.939510 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:03.939435    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:53:13.939622 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:13.939583    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:53:23.939516 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:23.939479    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:53:29.199064 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.199040    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:53:29.318607 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.318582    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc6r4\" (UniqueName: \"kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4\") pod \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") "
Sep 16 17:53:29.318743 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.318641    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs\") pod \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\" (UID: \"d5bb4e1d-af36-4378-bfe2-586aefd598f0\") "
Sep 16 17:53:29.320729 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.320692    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5bb4e1d-af36-4378-bfe2-586aefd598f0" (UID: "d5bb4e1d-af36-4378-bfe2-586aefd598f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:53:29.320842 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.320732    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4" (OuterVolumeSpecName: "kube-api-access-qc6r4") pod "d5bb4e1d-af36-4378-bfe2-586aefd598f0" (UID: "d5bb4e1d-af36-4378-bfe2-586aefd598f0"). InnerVolumeSpecName "kube-api-access-qc6r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:53:29.420203 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.420173    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc6r4\" (UniqueName: \"kubernetes.io/projected/d5bb4e1d-af36-4378-bfe2-586aefd598f0-kube-api-access-qc6r4\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:53:29.420203 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.420202    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5bb4e1d-af36-4378-bfe2-586aefd598f0-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:53:29.940389 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.940352    2559 generic.go:356] "Generic (PLEG): container finished" podID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerID="9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591" exitCode=0
Sep 16 17:53:29.940591 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.940406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" event={"ID":"d5bb4e1d-af36-4378-bfe2-586aefd598f0","Type":"ContainerDied","Data":"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"}
Sep 16 17:53:29.940591 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.940418    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"
Sep 16 17:53:29.940591 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.940447    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx" event={"ID":"d5bb4e1d-af36-4378-bfe2-586aefd598f0","Type":"ContainerDied","Data":"6059a575a08e5367dfbd528e76824e5a5a35d0252ef41874b429cfb987b0000f"}
Sep 16 17:53:29.940591 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.940468    2559 scope.go:117] "RemoveContainer" containerID="9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"
Sep 16 17:53:29.949737 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.949720    2559 scope.go:117] "RemoveContainer" containerID="9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"
Sep 16 17:53:29.949971 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:53:29.949953    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591\": container with ID starting with 9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591 not found: ID does not exist" containerID="9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"
Sep 16 17:53:29.950008 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.949980    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591"} err="failed to get container status \"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591\": rpc error: code = NotFound desc = could not find container \"9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591\": container with ID starting with 9343101f6d23e262180a67da2f34ddc256cdabe0e7d409e06e37f207609a4591 not found: ID does not exist"
Sep 16 17:53:29.965124 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.965099    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:53:29.969207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:29.969184    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5gkgwx"]
Sep 16 17:53:31.295436 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:31.295402    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" path="/var/lib/kubelet/pods/d5bb4e1d-af36-4378-bfe2-586aefd598f0/volumes"
Sep 16 17:53:36.415949 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.415919    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:53:36.416301 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.416270    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main"
Sep 16 17:53:36.416301 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.416287    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main"
Sep 16 17:53:36.416367 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.416356    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5bb4e1d-af36-4378-bfe2-586aefd598f0" containerName="main"
Sep 16 17:53:36.424170 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.424145    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.428328 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.428305    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Sep 16 17:53:36.428473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.428328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Sep 16 17:53:36.428473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.428351    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Sep 16 17:53:36.428795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.428779    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-smzct\""
Sep 16 17:53:36.431442 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.431423    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-epp-sa-dockercfg-wtw26\""
Sep 16 17:53:36.442528 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.442507    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:53:36.476334 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.476310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zrjb\" (UniqueName: \"kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.476420 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.476343    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.476420 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.476372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.476502 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.476465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.576799 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.576774    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.576899 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.576813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.576899 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.576845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.576974 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.576911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zrjb\" (UniqueName: \"kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.577163 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.577144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.577207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.577167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.579295 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.579270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.592948 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.592927    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zrjb\" (UniqueName: \"kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:36.734789 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:36.734722    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:53:37.037880 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:37.037843    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:53:37.040258 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:53:37.040230    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8820b437_0d53_42a8_ab04_ba5841fc28eb.slice/crio-e9ce69fa253700e5ec9c99833505e1db1b25af24856d5eb73a9771b09654823c WatchSource:0}: Error finding container e9ce69fa253700e5ec9c99833505e1db1b25af24856d5eb73a9771b09654823c: Status 404 returned error can't find the container with id e9ce69fa253700e5ec9c99833505e1db1b25af24856d5eb73a9771b09654823c
Sep 16 17:53:37.983040 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:37.983007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" event={"ID":"8820b437-0d53-42a8-ab04-ba5841fc28eb","Type":"ContainerStarted","Data":"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"}
Sep 16 17:53:37.983384 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:37.983057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" event={"ID":"8820b437-0d53-42a8-ab04-ba5841fc28eb","Type":"ContainerStarted","Data":"e9ce69fa253700e5ec9c99833505e1db1b25af24856d5eb73a9771b09654823c"}
Sep 16 17:53:37.983384 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:53:37.983096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:54:08.989954 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:08.989919    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:54:09.019740 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:09.019682    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podStartSLOduration=33.019661828 podStartE2EDuration="33.019661828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:53:38.024910317 +0000 UTC m=+1741.254692008" watchObservedRunningTime="2026-09-16 17:54:09.019661828 +0000 UTC m=+1772.249443532"
Sep 16 17:54:10.744247 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:10.744215    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:54:18.989857 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:18.989818    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:54:25.763874 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:25.763830    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main" containerID="cri-o://f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600" gracePeriod=45
Sep 16 17:54:28.990288 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:28.990248    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:54:37.340714 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:37.340688    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:54:37.344487 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:37.344461    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:54:38.990375 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:38.990339    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:54:38.990795 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:38.990448    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:54:48.990468 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:48.990428    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:54:56.005099 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.005074    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:54:56.085650 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache\") pod \"8820b437-0d53-42a8-ab04-ba5841fc28eb\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") "
Sep 16 17:54:56.085781 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085681    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs\") pod \"8820b437-0d53-42a8-ab04-ba5841fc28eb\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") "
Sep 16 17:54:56.085781 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085723    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache\") pod \"8820b437-0d53-42a8-ab04-ba5841fc28eb\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") "
Sep 16 17:54:56.085887 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085823    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zrjb\" (UniqueName: \"kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb\") pod \"8820b437-0d53-42a8-ab04-ba5841fc28eb\" (UID: \"8820b437-0d53-42a8-ab04-ba5841fc28eb\") "
Sep 16 17:54:56.085887 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085868    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "8820b437-0d53-42a8-ab04-ba5841fc28eb" (UID: "8820b437-0d53-42a8-ab04-ba5841fc28eb"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:54:56.086334 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.085999    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "8820b437-0d53-42a8-ab04-ba5841fc28eb" (UID: "8820b437-0d53-42a8-ab04-ba5841fc28eb"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:54:56.086334 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.086089    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-tokenizer-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:54:56.086334 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.086101    2559 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/8820b437-0d53-42a8-ab04-ba5841fc28eb-cachi2-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:54:56.087770 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.087735    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8820b437-0d53-42a8-ab04-ba5841fc28eb" (UID: "8820b437-0d53-42a8-ab04-ba5841fc28eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:54:56.087864 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.087822    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb" (OuterVolumeSpecName: "kube-api-access-8zrjb") pod "8820b437-0d53-42a8-ab04-ba5841fc28eb" (UID: "8820b437-0d53-42a8-ab04-ba5841fc28eb"). InnerVolumeSpecName "kube-api-access-8zrjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:54:56.187397 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.187375    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8820b437-0d53-42a8-ab04-ba5841fc28eb-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:54:56.187493 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.187398    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zrjb\" (UniqueName: \"kubernetes.io/projected/8820b437-0d53-42a8-ab04-ba5841fc28eb-kube-api-access-8zrjb\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:54:56.360962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.360906    2559 generic.go:356] "Generic (PLEG): container finished" podID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerID="f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600" exitCode=0
Sep 16 17:54:56.360962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.360945    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"
Sep 16 17:54:56.360962 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.360956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" event={"ID":"8820b437-0d53-42a8-ab04-ba5841fc28eb","Type":"ContainerDied","Data":"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"}
Sep 16 17:54:56.361141 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.360993    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf" event={"ID":"8820b437-0d53-42a8-ab04-ba5841fc28eb","Type":"ContainerDied","Data":"e9ce69fa253700e5ec9c99833505e1db1b25af24856d5eb73a9771b09654823c"}
Sep 16 17:54:56.361141 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.361013    2559 scope.go:117] "RemoveContainer" containerID="f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"
Sep 16 17:54:56.369747 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.369733    2559 scope.go:117] "RemoveContainer" containerID="f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"
Sep 16 17:54:56.369980 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:54:56.369962    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600\": container with ID starting with f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600 not found: ID does not exist" containerID="f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"
Sep 16 17:54:56.370027 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.369987    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600"} err="failed to get container status \"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600\": rpc error: code = NotFound desc = could not find container \"f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600\": container with ID starting with f7019d023ca290958385d933e6f57b32b812544a466956f034cb932996b43600 not found: ID does not exist"
Sep 16 17:54:56.389186 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.389160    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:54:56.396778 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:56.396759    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffcr5nrf"]
Sep 16 17:54:57.294981 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:54:57.294951    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" path="/var/lib/kubelet/pods/8820b437-0d53-42a8-ab04-ba5841fc28eb/volumes"
Sep 16 17:55:20.758702 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.758663    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:55:20.759116 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.759027    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main"
Sep 16 17:55:20.759116 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.759040    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main"
Sep 16 17:55:20.759116 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.759109    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8820b437-0d53-42a8-ab04-ba5841fc28eb" containerName="main"
Sep 16 17:55:20.762011 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.761992    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.766113 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.766097    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Sep 16 17:55:20.766349 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.766328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-vtflj\""
Sep 16 17:55:20.766444 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.766359    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-9lb7z\""
Sep 16 17:55:20.766444 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.766328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Sep 16 17:55:20.766549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.766455    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Sep 16 17:55:20.780850 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.780828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:55:20.877735 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.877707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8jsx\" (UniqueName: \"kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.877919 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.877836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.979084 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.979059    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8jsx\" (UniqueName: \"kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.979207 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.979119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.981388 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.981369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:20.988074 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:20.988053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8jsx\" (UniqueName: \"kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:21.072032 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:21.072009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:21.356069 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:21.355980    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:55:21.358152 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:55:21.358127    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc190e43a_bf4f_4628_bada_ce640b69a3a0.slice/crio-45c3e36a9bfb4695ffc2de005f1ea4c493f8385b8ecaedf1babf80e1193fd6d5 WatchSource:0}: Error finding container 45c3e36a9bfb4695ffc2de005f1ea4c493f8385b8ecaedf1babf80e1193fd6d5: Status 404 returned error can't find the container with id 45c3e36a9bfb4695ffc2de005f1ea4c493f8385b8ecaedf1babf80e1193fd6d5
Sep 16 17:55:21.472374 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:21.472342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" event={"ID":"c190e43a-bf4f-4628-bada-ce640b69a3a0","Type":"ContainerStarted","Data":"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"}
Sep 16 17:55:21.472538 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:21.472394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" event={"ID":"c190e43a-bf4f-4628-bada-ce640b69a3a0","Type":"ContainerStarted","Data":"45c3e36a9bfb4695ffc2de005f1ea4c493f8385b8ecaedf1babf80e1193fd6d5"}
Sep 16 17:55:21.472538 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:21.472423    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:52.479706 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:52.479678    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:55:52.522302 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:55:52.522261    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podStartSLOduration=32.522247098 podStartE2EDuration="32.522247098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:55:21.517667724 +0000 UTC m=+1844.747449415" watchObservedRunningTime="2026-09-16 17:55:52.522247098 +0000 UTC m=+1875.752028786"
Sep 16 17:57:24.929904 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:24.929864    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:57:32.479105 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:32.479021    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:57:39.949356 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:39.949310    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main" containerID="cri-o://9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58" gracePeriod=45
Sep 16 17:57:42.479761 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:42.479721    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:57:52.479748 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:52.479708    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:57:52.480232 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:57:52.479823    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:58:02.479498 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:02.479452    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:58:10.187318 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.187295    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:58:10.250602 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.250515    2559 generic.go:356] "Generic (PLEG): container finished" podID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerID="9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58" exitCode=0
Sep 16 17:58:10.250602 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.250577    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"
Sep 16 17:58:10.250602 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.250591    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" event={"ID":"c190e43a-bf4f-4628-bada-ce640b69a3a0","Type":"ContainerDied","Data":"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"}
Sep 16 17:58:10.250792 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.250640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb" event={"ID":"c190e43a-bf4f-4628-bada-ce640b69a3a0","Type":"ContainerDied","Data":"45c3e36a9bfb4695ffc2de005f1ea4c493f8385b8ecaedf1babf80e1193fd6d5"}
Sep 16 17:58:10.250792 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.250662    2559 scope.go:117] "RemoveContainer" containerID="9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"
Sep 16 17:58:10.258589 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.258574    2559 scope.go:117] "RemoveContainer" containerID="9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"
Sep 16 17:58:10.258873 ip-10-0-142-164 kubenswrapper[2559]: E0916 17:58:10.258854    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58\": container with ID starting with 9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58 not found: ID does not exist" containerID="9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"
Sep 16 17:58:10.258957 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.258882    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58"} err="failed to get container status \"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58\": rpc error: code = NotFound desc = could not find container \"9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58\": container with ID starting with 9d8474a3a0152b3951e259dfdbab9fa85a542705aba67712ea73335e69ac7e58 not found: ID does not exist"
Sep 16 17:58:10.322473 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.322453    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8jsx\" (UniqueName: \"kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx\") pod \"c190e43a-bf4f-4628-bada-ce640b69a3a0\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") "
Sep 16 17:58:10.322593 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.322495    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs\") pod \"c190e43a-bf4f-4628-bada-ce640b69a3a0\" (UID: \"c190e43a-bf4f-4628-bada-ce640b69a3a0\") "
Sep 16 17:58:10.324549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.324521    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c190e43a-bf4f-4628-bada-ce640b69a3a0" (UID: "c190e43a-bf4f-4628-bada-ce640b69a3a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:58:10.324549 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.324541    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx" (OuterVolumeSpecName: "kube-api-access-g8jsx") pod "c190e43a-bf4f-4628-bada-ce640b69a3a0" (UID: "c190e43a-bf4f-4628-bada-ce640b69a3a0"). InnerVolumeSpecName "kube-api-access-g8jsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:58:10.423633 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.423604    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8jsx\" (UniqueName: \"kubernetes.io/projected/c190e43a-bf4f-4628-bada-ce640b69a3a0-kube-api-access-g8jsx\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:58:10.423633 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.423635    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c190e43a-bf4f-4628-bada-ce640b69a3a0-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 17:58:10.573346 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.573316    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:58:10.577401 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:10.577381    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75k9xbb"]
Sep 16 17:58:11.294946 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:58:11.294907    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" path="/var/lib/kubelet/pods/c190e43a-bf4f-4628-bada-ce640b69a3a0/volumes"
Sep 16 17:59:18.678703 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.678667    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 17:59:18.679127 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.679059    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main"
Sep 16 17:59:18.679127 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.679075    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main"
Sep 16 17:59:18.679227 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.679174    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c190e43a-bf4f-4628-bada-ce640b69a3a0" containerName="main"
Sep 16 17:59:18.682123 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.682104    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.685398 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.685378    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"kube-root-ca.crt\""
Sep 16 17:59:18.685547 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.685532    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"stop-v1-kserve-self-signed-certs\""
Sep 16 17:59:18.686758 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.686735    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"openshift-service-ca.crt\""
Sep 16 17:59:18.687037 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.687019    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"default-dockercfg-2zlh7\""
Sep 16 17:59:18.696947 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.696927    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 17:59:18.768961 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.768940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.769071 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.768976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.769071 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.769005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.769148 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.769093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.769187 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.769149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.769221 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.769188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8sdk\" (UniqueName: \"kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.869833 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.869947 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8sdk\" (UniqueName: \"kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.869947 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.869947 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.869947 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869921    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.870162 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.869941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.870299 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.870270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.870425 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.870277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.870425 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.870346    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.872023 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.872003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.872273 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.872257    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.883546 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.883525    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8sdk\" (UniqueName: \"kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk\") pod \"stop-v1-kserve-5657b88c95-65kj8\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:18.992604 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:18.992535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:19.272453 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:19.272420    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 17:59:19.273918 ip-10-0-142-164 kubenswrapper[2559]: W0916 17:59:19.273892    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a63a16a_c6ce_4894_8e63_353704112e71.slice/crio-cfd9d5c5a3472813fd2c8446f699f3a7693a8e54e3fb968fa1ac924405494616 WatchSource:0}: Error finding container cfd9d5c5a3472813fd2c8446f699f3a7693a8e54e3fb968fa1ac924405494616: Status 404 returned error can't find the container with id cfd9d5c5a3472813fd2c8446f699f3a7693a8e54e3fb968fa1ac924405494616
Sep 16 17:59:19.276130 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:19.276114    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:59:19.556147 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:19.556068    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" event={"ID":"4a63a16a-c6ce-4894-8e63-353704112e71","Type":"ContainerStarted","Data":"cfd9d5c5a3472813fd2c8446f699f3a7693a8e54e3fb968fa1ac924405494616"}
Sep 16 17:59:20.562271 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:20.562185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" event={"ID":"4a63a16a-c6ce-4894-8e63-353704112e71","Type":"ContainerStarted","Data":"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"}
Sep 16 17:59:20.597404 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:20.597356    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" podStartSLOduration=1.6449112449999999 podStartE2EDuration="2.597342499s" totalImagesPullingTime="952.431254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:59:20.594528012 +0000 UTC m=+2083.824309704" watchObservedRunningTime="2026-09-16 17:59:20.597342499 +0000 UTC m=+2083.827124189"
Sep 16 17:59:20.993646 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:20.993615    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:28.993316 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:28.993277    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:29.005439 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:29.005418    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:29.616034 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:29.616004    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 17:59:37.370392 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:37.370360    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 17:59:37.373150 ip-10-0-142-164 kubenswrapper[2559]: I0916 17:59:37.373128    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:00:30.089115 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:30.089042    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 18:00:45.107303 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.107262    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" podUID="4a63a16a-c6ce-4894-8e63-353704112e71" containerName="main" containerID="cri-o://05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355" gracePeriod=45
Sep 16 18:00:45.353381 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.353356    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 18:00:45.446229 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446147    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446229 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446205    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446229 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446222    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446243    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446282    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446322    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8sdk\" (UniqueName: \"kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk\") pod \"4a63a16a-c6ce-4894-8e63-353704112e71\" (UID: \"4a63a16a-c6ce-4894-8e63-353704112e71\") "
Sep 16 18:00:45.446659 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446493    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:00:45.446659 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446621    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home" (OuterVolumeSpecName: "home") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:00:45.446659 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:00:45.446833 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446733    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.446833 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446755    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.446833 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.446768    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.448608 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.448580    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:00:45.448711 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.448588    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm" (OuterVolumeSpecName: "dshm") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:00:45.448711 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.448684    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk" (OuterVolumeSpecName: "kube-api-access-x8sdk") pod "4a63a16a-c6ce-4894-8e63-353704112e71" (UID: "4a63a16a-c6ce-4894-8e63-353704112e71"). InnerVolumeSpecName "kube-api-access-x8sdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:00:45.547424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.547395    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a63a16a-c6ce-4894-8e63-353704112e71-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.547424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.547425    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a63a16a-c6ce-4894-8e63-353704112e71-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.547658 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.547438    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8sdk\" (UniqueName: \"kubernetes.io/projected/4a63a16a-c6ce-4894-8e63-353704112e71-kube-api-access-x8sdk\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:00:45.952591 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.952520    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a63a16a-c6ce-4894-8e63-353704112e71" containerID="05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355" exitCode=0
Sep 16 18:00:45.952773 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.952597    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"
Sep 16 18:00:45.952773 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.952597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" event={"ID":"4a63a16a-c6ce-4894-8e63-353704112e71","Type":"ContainerDied","Data":"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"}
Sep 16 18:00:45.952773 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.952669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8" event={"ID":"4a63a16a-c6ce-4894-8e63-353704112e71","Type":"ContainerDied","Data":"cfd9d5c5a3472813fd2c8446f699f3a7693a8e54e3fb968fa1ac924405494616"}
Sep 16 18:00:45.952773 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.952703    2559 scope.go:117] "RemoveContainer" containerID="05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"
Sep 16 18:00:45.961657 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.961640    2559 scope.go:117] "RemoveContainer" containerID="05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"
Sep 16 18:00:45.961933 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:00:45.961915    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355\": container with ID starting with 05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355 not found: ID does not exist" containerID="05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"
Sep 16 18:00:45.962022 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.961943    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355"} err="failed to get container status \"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355\": rpc error: code = NotFound desc = could not find container \"05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355\": container with ID starting with 05cc49afbc2e33543dbc7d5221578386d81b75f7d55ef7dab0ef879db6894355 not found: ID does not exist"
Sep 16 18:00:45.974714 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.974693    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 18:00:45.980440 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:45.980420    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-65kj8"]
Sep 16 18:00:47.300606 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:47.300551    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a63a16a-c6ce-4894-8e63-353704112e71" path="/var/lib/kubelet/pods/4a63a16a-c6ce-4894-8e63-353704112e71/volumes"
Sep 16 18:00:59.998279 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:59.998247    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:00:59.998734 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:59.998715    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a63a16a-c6ce-4894-8e63-353704112e71" containerName="main"
Sep 16 18:00:59.998789 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:59.998737    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a63a16a-c6ce-4894-8e63-353704112e71" containerName="main"
Sep 16 18:00:59.998825 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:00:59.998816    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a63a16a-c6ce-4894-8e63-353704112e71" containerName="main"
Sep 16 18:01:00.003191 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.003170    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.007240 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.007204    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Sep 16 18:01:00.007240 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.007233    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-8xfng\""
Sep 16 18:01:00.007501 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.007485    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Sep 16 18:01:00.007590 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.007500    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Sep 16 18:01:00.007590 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.007500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-wmsfk\""
Sep 16 18:01:00.016341 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.016322    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:01:00.163899 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.163857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdnq7\" (UniqueName: \"kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.164088 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.164007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.264757 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.264657    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdnq7\" (UniqueName: \"kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.264933 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.264768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.268034 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.267994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.273626 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.273598    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdnq7\" (UniqueName: \"kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.314186 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.314156    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:00.594213 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:00.594186    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:01:00.595800 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:01:00.595772    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7ee21f_42e8_4c99_b94d_80d5a26b5d22.slice/crio-04505ef93628ebcb30cd9c5d6dffdb03167c1a5dc124c1985d90ab79d4c4fa5b WatchSource:0}: Error finding container 04505ef93628ebcb30cd9c5d6dffdb03167c1a5dc124c1985d90ab79d4c4fa5b: Status 404 returned error can't find the container with id 04505ef93628ebcb30cd9c5d6dffdb03167c1a5dc124c1985d90ab79d4c4fa5b
Sep 16 18:01:01.021108 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:01.021018    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" event={"ID":"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22","Type":"ContainerStarted","Data":"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"}
Sep 16 18:01:01.021108 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:01.021071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" event={"ID":"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22","Type":"ContainerStarted","Data":"04505ef93628ebcb30cd9c5d6dffdb03167c1a5dc124c1985d90ab79d4c4fa5b"}
Sep 16 18:01:01.021528 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:01.021134    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:01:01.048070 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:01.048020    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podStartSLOduration=2.048004801 podStartE2EDuration="2.048004801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:01:01.046484975 +0000 UTC m=+2184.276266679" watchObservedRunningTime="2026-09-16 18:01:01.048004801 +0000 UTC m=+2184.277786491"
Sep 16 18:01:32.029461 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:01:32.029431    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:03:02.029692 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:02.029653    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:03:02.258424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:02.258393    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:03:12.029511 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:12.029473    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:03:17.277199 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:17.277158    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" containerID="cri-o://f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4" gracePeriod=45
Sep 16 18:03:22.029943 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:22.029899    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:03:22.030332 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:22.030013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:03:32.030106 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:32.030012    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:03:42.029414 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:42.029369    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:03:47.530166 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.530142    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:03:47.574844 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.574816    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdnq7\" (UniqueName: \"kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7\") pod \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") "
Sep 16 18:03:47.574964 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.574950    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs\") pod \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\" (UID: \"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22\") "
Sep 16 18:03:47.576896 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.576866    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7" (OuterVolumeSpecName: "kube-api-access-cdnq7") pod "3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" (UID: "3e7ee21f-42e8-4c99-b94d-80d5a26b5d22"). InnerVolumeSpecName "kube-api-access-cdnq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:03:47.576986 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.576914    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" (UID: "3e7ee21f-42e8-4c99-b94d-80d5a26b5d22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:03:47.676350 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.676323    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:03:47.676454 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.676352    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdnq7\" (UniqueName: \"kubernetes.io/projected/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22-kube-api-access-cdnq7\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:03:47.782887 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.782860    2559 generic.go:356] "Generic (PLEG): container finished" podID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerID="f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4" exitCode=0
Sep 16 18:03:47.783027 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.782902    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"
Sep 16 18:03:47.783027 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.782952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" event={"ID":"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22","Type":"ContainerDied","Data":"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"}
Sep 16 18:03:47.783027 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.783011    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl" event={"ID":"3e7ee21f-42e8-4c99-b94d-80d5a26b5d22","Type":"ContainerDied","Data":"04505ef93628ebcb30cd9c5d6dffdb03167c1a5dc124c1985d90ab79d4c4fa5b"}
Sep 16 18:03:47.783209 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.783043    2559 scope.go:117] "RemoveContainer" containerID="f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"
Sep 16 18:03:47.791981 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.791965    2559 scope.go:117] "RemoveContainer" containerID="f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"
Sep 16 18:03:47.792276 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:03:47.792257    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4\": container with ID starting with f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4 not found: ID does not exist" containerID="f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"
Sep 16 18:03:47.792360 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.792283    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4"} err="failed to get container status \"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4\": rpc error: code = NotFound desc = could not find container \"f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4\": container with ID starting with f6cd820fbcd987a9ea3344f38f9de50dcff1e43e88a7ec1b1bc6ab1c6e845cb4 not found: ID does not exist"
Sep 16 18:03:47.818861 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.818839    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:03:47.838253 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:47.838200    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b694kznl"]
Sep 16 18:03:49.295620 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:03:49.295580    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" path="/var/lib/kubelet/pods/3e7ee21f-42e8-4c99-b94d-80d5a26b5d22/volumes"
Sep 16 18:04:37.404411 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:37.404380    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:04:37.408354 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:37.408334    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:04:55.207201 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.207164    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:04:55.207826 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.207534    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main"
Sep 16 18:04:55.207826 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.207545    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main"
Sep 16 18:04:55.207826 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.207631    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7ee21f-42e8-4c99-b94d-80d5a26b5d22" containerName="main"
Sep 16 18:04:55.210519 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.210492    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.214368 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.214340    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-rhvtt\""
Sep 16 18:04:55.215241 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.215220    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Sep 16 18:04:55.215798 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.215775    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-7sm9c\""
Sep 16 18:04:55.215907 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.215795    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Sep 16 18:04:55.215907 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.215799    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Sep 16 18:04:55.226891 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.226854    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:04:55.305455 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.305416    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzz8n\" (UniqueName: \"kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.305692 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.305491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.406317 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.406270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz8n\" (UniqueName: \"kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.406547 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.406358    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.409093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.409065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.417193 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.417161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzz8n\" (UniqueName: \"kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.522291 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.522176    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:55.817508 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:55.817479    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:04:55.819645 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:04:55.819606    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f1f765_fee0_4f3d_9dc9_e7887541b963.slice/crio-551e7416ff24b0ef35abe70f126dd8b52ff99c5c84852391c92267ea2db24ebb WatchSource:0}: Error finding container 551e7416ff24b0ef35abe70f126dd8b52ff99c5c84852391c92267ea2db24ebb: Status 404 returned error can't find the container with id 551e7416ff24b0ef35abe70f126dd8b52ff99c5c84852391c92267ea2db24ebb
Sep 16 18:04:56.100204 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:56.100105    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" event={"ID":"29f1f765-fee0-4f3d-9dc9-e7887541b963","Type":"ContainerStarted","Data":"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"}
Sep 16 18:04:56.100204 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:56.100164    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" event={"ID":"29f1f765-fee0-4f3d-9dc9-e7887541b963","Type":"ContainerStarted","Data":"551e7416ff24b0ef35abe70f126dd8b52ff99c5c84852391c92267ea2db24ebb"}
Sep 16 18:04:56.100401 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:56.100206    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:04:56.130452 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:04:56.130394    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podStartSLOduration=1.130372124 podStartE2EDuration="1.130372124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:04:56.127491168 +0000 UTC m=+2419.357272860" watchObservedRunningTime="2026-09-16 18:04:56.130372124 +0000 UTC m=+2419.360153815"
Sep 16 18:05:27.106789 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:05:27.106758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:05:39.526616 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:05:39.526577    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:05:47.107330 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:05:47.107288    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:05:54.547006 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:05:54.546959    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main" containerID="cri-o://bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9" gracePeriod=45
Sep 16 18:05:57.106603 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:05:57.106539    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:06:07.107248 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:07.107208    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:06:07.107758 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:07.107315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:06:17.107177 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:17.107127    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:06:24.801354 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:24.801329    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:06:24.938590 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:24.938544    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs\") pod \"29f1f765-fee0-4f3d-9dc9-e7887541b963\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") "
Sep 16 18:06:24.938763 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:24.938644    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzz8n\" (UniqueName: \"kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n\") pod \"29f1f765-fee0-4f3d-9dc9-e7887541b963\" (UID: \"29f1f765-fee0-4f3d-9dc9-e7887541b963\") "
Sep 16 18:06:24.940684 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:24.940652    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n" (OuterVolumeSpecName: "kube-api-access-hzz8n") pod "29f1f765-fee0-4f3d-9dc9-e7887541b963" (UID: "29f1f765-fee0-4f3d-9dc9-e7887541b963"). InnerVolumeSpecName "kube-api-access-hzz8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:06:24.940868 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:24.940841    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29f1f765-fee0-4f3d-9dc9-e7887541b963" (UID: "29f1f765-fee0-4f3d-9dc9-e7887541b963"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:06:25.039239 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.039207    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29f1f765-fee0-4f3d-9dc9-e7887541b963-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:06:25.039239 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.039234    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzz8n\" (UniqueName: \"kubernetes.io/projected/29f1f765-fee0-4f3d-9dc9-e7887541b963-kube-api-access-hzz8n\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:06:25.552210 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.552173    2559 generic.go:356] "Generic (PLEG): container finished" podID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerID="bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9" exitCode=0
Sep 16 18:06:25.552382 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.552218    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"
Sep 16 18:06:25.552382 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.552235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" event={"ID":"29f1f765-fee0-4f3d-9dc9-e7887541b963","Type":"ContainerDied","Data":"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"}
Sep 16 18:06:25.552382 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.552286    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh" event={"ID":"29f1f765-fee0-4f3d-9dc9-e7887541b963","Type":"ContainerDied","Data":"551e7416ff24b0ef35abe70f126dd8b52ff99c5c84852391c92267ea2db24ebb"}
Sep 16 18:06:25.552382 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.552309    2559 scope.go:117] "RemoveContainer" containerID="bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"
Sep 16 18:06:25.560522 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.560501    2559 scope.go:117] "RemoveContainer" containerID="bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"
Sep 16 18:06:25.560839 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:06:25.560817    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9\": container with ID starting with bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9 not found: ID does not exist" containerID="bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"
Sep 16 18:06:25.560892 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.560847    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9"} err="failed to get container status \"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9\": rpc error: code = NotFound desc = could not find container \"bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9\": container with ID starting with bfdafa594b967a331225d0c3e4aa6bd8039b74f36a8cce423501b096779068a9 not found: ID does not exist"
Sep 16 18:06:25.585179 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.585148    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:06:25.591863 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:25.591831    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785v9sbh"]
Sep 16 18:06:27.295150 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:06:27.295117    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" path="/var/lib/kubelet/pods/29f1f765-fee0-4f3d-9dc9-e7887541b963/volumes"
Sep 16 18:07:15.571762 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.571728    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:07:15.572221 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.572069    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main"
Sep 16 18:07:15.572221 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.572080    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main"
Sep 16 18:07:15.572221 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.572150    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="29f1f765-fee0-4f3d-9dc9-e7887541b963" containerName="main"
Sep 16 18:07:15.574848 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.574832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.581453 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.581432    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-late-join-483adbac\"/\"kube-root-ca.crt\""
Sep 16 18:07:15.582424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.582404    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-late-join-483adbac\"/\"late-v2-kserve-self-signed-certs\""
Sep 16 18:07:15.582424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.582421    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-late-join-483adbac\"/\"default-dockercfg-l4pm2\""
Sep 16 18:07:15.582613 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.582405    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-late-join-483adbac\"/\"openshift-service-ca.crt\""
Sep 16 18:07:15.606400 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.606374    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:07:15.710236 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zbtl\" (UniqueName: \"kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.710395 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.710395 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.710395 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.710509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.710509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.710435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811808 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zbtl\" (UniqueName: \"kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811976 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811976 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811976 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811976 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.811976 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.811915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.812249 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.812227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.812316 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.812280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.812375 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.812360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.814227 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.814199    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.814618 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.814591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.820712 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.820690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zbtl\" (UniqueName: \"kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl\") pod \"late-v2-kserve-86766546f-jdg62\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") " pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:15.884740 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:15.884687    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:16.165126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:16.165103    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:07:16.166827 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:07:16.166802    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a16654_4b17_476c_a180_cd00a0649e63.slice/crio-f623f19142ee799c4c6abbd798e6d2445a74c9302a832959c7d028dc0fd40bf9 WatchSource:0}: Error finding container f623f19142ee799c4c6abbd798e6d2445a74c9302a832959c7d028dc0fd40bf9: Status 404 returned error can't find the container with id f623f19142ee799c4c6abbd798e6d2445a74c9302a832959c7d028dc0fd40bf9
Sep 16 18:07:16.786778 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:16.786742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" event={"ID":"82a16654-4b17-476c-a180-cd00a0649e63","Type":"ContainerStarted","Data":"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"}
Sep 16 18:07:16.787109 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:16.786800    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" event={"ID":"82a16654-4b17-476c-a180-cd00a0649e63","Type":"ContainerStarted","Data":"f623f19142ee799c4c6abbd798e6d2445a74c9302a832959c7d028dc0fd40bf9"}
Sep 16 18:07:16.821140 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:16.821073    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" podStartSLOduration=1.821054877 podStartE2EDuration="1.821054877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:07:16.818046897 +0000 UTC m=+2560.047828587" watchObservedRunningTime="2026-09-16 18:07:16.821054877 +0000 UTC m=+2560.050836569"
Sep 16 18:07:16.885798 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:16.885768    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:25.885100 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:25.885064    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:25.897684 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:25.897653    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:07:26.846472 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:07:26.846445    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:08:35.506380 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:35.506337    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:08:50.525444 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.525404    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" podUID="82a16654-4b17-476c-a180-cd00a0649e63" containerName="main" containerID="cri-o://dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a" gracePeriod=45
Sep 16 18:08:50.778027 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.777965    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:08:50.961685 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961652    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.961685 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.961900 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961732    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.961900 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961776    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.961900 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961824    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zbtl\" (UniqueName: \"kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.961900 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961861    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs\") pod \"82a16654-4b17-476c-a180-cd00a0649e63\" (UID: \"82a16654-4b17-476c-a180-cd00a0649e63\") "
Sep 16 18:08:50.962128 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.961982    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:08:50.962128 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.962018    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home" (OuterVolumeSpecName: "home") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:08:50.962128 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.962057    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache" (OuterVolumeSpecName: "model-cache") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:08:50.962292 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.962203    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:50.962292 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.962217    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:50.962292 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.962226    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:50.964077 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.964042    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl" (OuterVolumeSpecName: "kube-api-access-5zbtl") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "kube-api-access-5zbtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:08:50.964187 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.964086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:08:50.964187 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:50.964089    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm" (OuterVolumeSpecName: "dshm") pod "82a16654-4b17-476c-a180-cd00a0649e63" (UID: "82a16654-4b17-476c-a180-cd00a0649e63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:08:51.063061 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.062984    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zbtl\" (UniqueName: \"kubernetes.io/projected/82a16654-4b17-476c-a180-cd00a0649e63-kube-api-access-5zbtl\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:51.063061 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.063013    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82a16654-4b17-476c-a180-cd00a0649e63-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:51.063061 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.063023    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82a16654-4b17-476c-a180-cd00a0649e63-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:08:51.223401 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.223368    2559 generic.go:356] "Generic (PLEG): container finished" podID="82a16654-4b17-476c-a180-cd00a0649e63" containerID="dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a" exitCode=0
Sep 16 18:08:51.223582 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.223414    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"
Sep 16 18:08:51.223582 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.223456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" event={"ID":"82a16654-4b17-476c-a180-cd00a0649e63","Type":"ContainerDied","Data":"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"}
Sep 16 18:08:51.223582 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.223514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62" event={"ID":"82a16654-4b17-476c-a180-cd00a0649e63","Type":"ContainerDied","Data":"f623f19142ee799c4c6abbd798e6d2445a74c9302a832959c7d028dc0fd40bf9"}
Sep 16 18:08:51.223582 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.223545    2559 scope.go:117] "RemoveContainer" containerID="dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"
Sep 16 18:08:51.232199 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.232178    2559 scope.go:117] "RemoveContainer" containerID="dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"
Sep 16 18:08:51.232452 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:08:51.232436    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a\": container with ID starting with dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a not found: ID does not exist" containerID="dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"
Sep 16 18:08:51.232507 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.232458    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a"} err="failed to get container status \"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a\": rpc error: code = NotFound desc = could not find container \"dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a\": container with ID starting with dea4a81dcd104e558a1307f9eb9e8b15110c3b28c1cd6b37437e5d2345b1b38a not found: ID does not exist"
Sep 16 18:08:51.248112 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.248090    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:08:51.252619 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.252597    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-late-join-483adbac/late-v2-kserve-86766546f-jdg62"]
Sep 16 18:08:51.295943 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:08:51.295920    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82a16654-4b17-476c-a180-cd00a0649e63" path="/var/lib/kubelet/pods/82a16654-4b17-476c-a180-cd00a0649e63/volumes"
Sep 16 18:09:37.433515 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:09:37.433433    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:09:37.438419 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:09:37.438392    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:10:50.220277 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.220237    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:10:50.220885 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.220843    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a16654-4b17-476c-a180-cd00a0649e63" containerName="main"
Sep 16 18:10:50.220885 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.220863    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a16654-4b17-476c-a180-cd00a0649e63" containerName="main"
Sep 16 18:10:50.221066 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.220948    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a16654-4b17-476c-a180-cd00a0649e63" containerName="main"
Sep 16 18:10:50.224026 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.224005    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.228694 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:10:50.228669    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-9qcfc\" is forbidden: User \"system:node:ip-10-0-142-164.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"e2e-test-rollback-b8fdb441\": no relationship found between node 'ip-10-0-142-164.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"default-dockercfg-9qcfc\"" type="*v1.Secret"
Sep 16 18:10:50.243178 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.243159    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"kube-root-ca.crt\""
Sep 16 18:10:50.243406 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.243389    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"openshift-service-ca.crt\""
Sep 16 18:10:50.244291 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.244276    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"roll-v1-kserve-self-signed-certs\""
Sep 16 18:10:50.337378 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.337347    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:10:50.338988 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.338962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.339096 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.339035    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9g5k\" (UniqueName: \"kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.339096 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.339072    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.339194 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.339125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.339194 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.339174    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.339270 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.339240    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.439889 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.439857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440039 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.439915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440039 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.439949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440039 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.439990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440039 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.440029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440244 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.440140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9g5k\" (UniqueName: \"kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440305 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.440288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440389 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.440367    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.440450 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.440402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.442390 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.442365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.442479 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.442421    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:50.456873 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:50.456848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9g5k\" (UniqueName: \"kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k\") pod \"roll-v1-kserve-69cf8bcfc-5btlf\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:51.535456 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:51.535422    2559 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 16 18:10:51.535851 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:51.535490    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:10:51.729688 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:51.729661    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"default-dockercfg-9qcfc\""
Sep 16 18:10:51.837794 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:51.837769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:10:51.839019 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:10:51.838988    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6925fe_0881_4157_a382_0f10a3d3c61f.slice/crio-8127d0d70f3cf6352fb3aaf77ab7615e23822b64a23bdb190e6c3726092a7b19 WatchSource:0}: Error finding container 8127d0d70f3cf6352fb3aaf77ab7615e23822b64a23bdb190e6c3726092a7b19: Status 404 returned error can't find the container with id 8127d0d70f3cf6352fb3aaf77ab7615e23822b64a23bdb190e6c3726092a7b19
Sep 16 18:10:52.803218 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:52.803178    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" event={"ID":"3b6925fe-0881-4157-a382-0f10a3d3c61f","Type":"ContainerStarted","Data":"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"}
Sep 16 18:10:52.803674 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:52.803234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" event={"ID":"3b6925fe-0881-4157-a382-0f10a3d3c61f","Type":"ContainerStarted","Data":"8127d0d70f3cf6352fb3aaf77ab7615e23822b64a23bdb190e6c3726092a7b19"}
Sep 16 18:10:52.825538 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:52.825487    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" podStartSLOduration=2.8254717769999997 podStartE2EDuration="2.825471777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:10:52.824009617 +0000 UTC m=+2776.053791307" watchObservedRunningTime="2026-09-16 18:10:52.825471777 +0000 UTC m=+2776.055253477"
Sep 16 18:10:53.535827 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:10:53.535797    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:11:01.535926 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:11:01.535841    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:11:01.548630 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:11:01.548594    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:11:01.856049 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:11:01.856019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:12:36.444554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.444475    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:12:36.448101 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.448085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.455036 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.455016    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-9t6sk\""
Sep 16 18:12:36.455147 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.455019    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-v98fj\""
Sep 16 18:12:36.456081 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.456063    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Sep 16 18:12:36.456358 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.456337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Sep 16 18:12:36.457424 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.457409    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Sep 16 18:12:36.466126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.466103    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:12:36.574568 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.574541    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.574694 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.574591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svtxt\" (UniqueName: \"kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.675694 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.675665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.675805 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.675701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svtxt\" (UniqueName: \"kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.678330 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.678300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.686344 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.686322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svtxt\" (UniqueName: \"kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:36.758189 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:36.758128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:37.037411 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:37.037382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:12:37.039138 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:12:37.039117    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc0f25f_a0c3_44c5_ae21_f0665f743ff2.slice/crio-5a4b8d23f04855761a52e93d257c053ff09b3dab47e8ef6864e29f2f1250c9b3 WatchSource:0}: Error finding container 5a4b8d23f04855761a52e93d257c053ff09b3dab47e8ef6864e29f2f1250c9b3: Status 404 returned error can't find the container with id 5a4b8d23f04855761a52e93d257c053ff09b3dab47e8ef6864e29f2f1250c9b3
Sep 16 18:12:37.287988 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:37.287913    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" event={"ID":"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2","Type":"ContainerStarted","Data":"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"}
Sep 16 18:12:37.287988 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:37.287961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" event={"ID":"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2","Type":"ContainerStarted","Data":"5a4b8d23f04855761a52e93d257c053ff09b3dab47e8ef6864e29f2f1250c9b3"}
Sep 16 18:12:37.288180 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:37.287994    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:12:37.312110 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:12:37.312067    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podStartSLOduration=1.312047 podStartE2EDuration="1.312047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:12:37.310407902 +0000 UTC m=+2880.540189595" watchObservedRunningTime="2026-09-16 18:12:37.312047 +0000 UTC m=+2880.541828725"
Sep 16 18:13:08.293705 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:08.293674    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:13:09.579508 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:09.579473    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:13:18.294035 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:18.293993    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:13:24.599400 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:24.599340    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main" containerID="cri-o://f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c" gracePeriod=45
Sep 16 18:13:28.293837 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:28.293798    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:13:38.294012 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:38.293970    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:13:38.294495 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:38.294103    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:13:41.776905 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:41.776873    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:13:48.294239 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:48.294196    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:13:54.844010 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:54.843986    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:13:54.915802 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:54.915747    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svtxt\" (UniqueName: \"kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt\") pod \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") "
Sep 16 18:13:54.915802 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:54.915800    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs\") pod \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\" (UID: \"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2\") "
Sep 16 18:13:54.917864 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:54.917837    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt" (OuterVolumeSpecName: "kube-api-access-svtxt") pod "1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" (UID: "1fc0f25f-a0c3-44c5-ae21-f0665f743ff2"). InnerVolumeSpecName "kube-api-access-svtxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:13:54.917961 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:54.917902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" (UID: "1fc0f25f-a0c3-44c5-ae21-f0665f743ff2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:13:55.016348 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.016323    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svtxt\" (UniqueName: \"kubernetes.io/projected/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-kube-api-access-svtxt\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:55.016348 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.016347    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:55.643902 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.643861    2559 generic.go:356] "Generic (PLEG): container finished" podID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerID="f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c" exitCode=0
Sep 16 18:13:55.643902 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.643898    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"
Sep 16 18:13:55.644093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.643926    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" event={"ID":"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2","Type":"ContainerDied","Data":"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"}
Sep 16 18:13:55.644093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.643982    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q" event={"ID":"1fc0f25f-a0c3-44c5-ae21-f0665f743ff2","Type":"ContainerDied","Data":"5a4b8d23f04855761a52e93d257c053ff09b3dab47e8ef6864e29f2f1250c9b3"}
Sep 16 18:13:55.644093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.644013    2559 scope.go:117] "RemoveContainer" containerID="f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"
Sep 16 18:13:55.652030 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.652014    2559 scope.go:117] "RemoveContainer" containerID="f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"
Sep 16 18:13:55.652292 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:13:55.652273    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c\": container with ID starting with f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c not found: ID does not exist" containerID="f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"
Sep 16 18:13:55.652341 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.652300    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c"} err="failed to get container status \"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c\": rpc error: code = NotFound desc = could not find container \"f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c\": container with ID starting with f24d58bae425138569c46d9c4b9ffe2e4ba49fdf8ba33ac08622b11175a8159c not found: ID does not exist"
Sep 16 18:13:55.667612 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.667592    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:13:55.678320 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:55.678300    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheph59q"]
Sep 16 18:13:56.455913 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.455885    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:13:56.456263 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.456223    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main"
Sep 16 18:13:56.456263 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.456234    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main"
Sep 16 18:13:56.456336 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.456297    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" containerName="main"
Sep 16 18:13:56.460503 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.460483    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.464874 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.464855    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"kube-root-ca.crt\""
Sep 16 18:13:56.464874 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.464860    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"openshift-service-ca.crt\""
Sep 16 18:13:56.465031 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.464856    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v1-kserve-self-signed-certs\""
Sep 16 18:13:56.465031 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.464877    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"default-dockercfg-q8zfv\""
Sep 16 18:13:56.473740 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.473720    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:13:56.527134 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.527255 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.527255 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527242    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.527356 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.527356 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbtdc\" (UniqueName: \"kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.527428 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.527371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628481 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628577    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628813 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbtdc\" (UniqueName: \"kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628813 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628929 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628888    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.628929 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.628921    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.629079 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.629060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.630887 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.630865    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.631216 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.631199    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.640238 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.640217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbtdc\" (UniqueName: \"kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc\") pod \"owner-v1-kserve-75579ff5bc-l2ck9\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.771650 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.771590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:56.795223 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.795195    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerName="main" containerID="cri-o://f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a" gracePeriod=45
Sep 16 18:13:56.845595 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:56.845547    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.39:8000/health\": dial tcp 10.132.0.39:8000: connect: connection refused"
Sep 16 18:13:57.026141 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.026088    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:13:57.052905 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.052880    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:13:57.055040 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:13:57.055017    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d91c42_5e44_48c3_b0d0_224963c0da07.slice/crio-39f67a085b07023c67358941977c05a3c4d177e6b79f725cd51c808f679d4134 WatchSource:0}: Error finding container 39f67a085b07023c67358941977c05a3c4d177e6b79f725cd51c808f679d4134: Status 404 returned error can't find the container with id 39f67a085b07023c67358941977c05a3c4d177e6b79f725cd51c808f679d4134
Sep 16 18:13:57.133420 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133394    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133523 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133482    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133523 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133504    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133651 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133527    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9g5k\" (UniqueName: \"kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133651 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133578    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133758 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133648    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:13:57.133815 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs\") pod \"3b6925fe-0881-4157-a382-0f10a3d3c61f\" (UID: \"3b6925fe-0881-4157-a382-0f10a3d3c61f\") "
Sep 16 18:13:57.133815 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133800    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:13:57.133926 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.133815    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home" (OuterVolumeSpecName: "home") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:13:57.134148 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.134124    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.134258 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.134151    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.134258 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.134164    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.135667 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.135644    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k" (OuterVolumeSpecName: "kube-api-access-d9g5k") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "kube-api-access-d9g5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:13:57.135774 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.135756    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm" (OuterVolumeSpecName: "dshm") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:13:57.136053 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.136031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b6925fe-0881-4157-a382-0f10a3d3c61f" (UID: "3b6925fe-0881-4157-a382-0f10a3d3c61f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:13:57.235247 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.235225    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b6925fe-0881-4157-a382-0f10a3d3c61f-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.235247 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.235246    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6925fe-0881-4157-a382-0f10a3d3c61f-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.235387 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.235257    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9g5k\" (UniqueName: \"kubernetes.io/projected/3b6925fe-0881-4157-a382-0f10a3d3c61f-kube-api-access-d9g5k\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:13:57.295124 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.295068    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc0f25f-a0c3-44c5-ae21-f0665f743ff2" path="/var/lib/kubelet/pods/1fc0f25f-a0c3-44c5-ae21-f0665f743ff2/volumes"
Sep 16 18:13:57.657089 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.657057    2559 generic.go:356] "Generic (PLEG): container finished" podID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerID="f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a" exitCode=0
Sep 16 18:13:57.657469 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.657099    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"
Sep 16 18:13:57.657469 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.657149    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" event={"ID":"3b6925fe-0881-4157-a382-0f10a3d3c61f","Type":"ContainerDied","Data":"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"}
Sep 16 18:13:57.657469 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.657197    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf" event={"ID":"3b6925fe-0881-4157-a382-0f10a3d3c61f","Type":"ContainerDied","Data":"8127d0d70f3cf6352fb3aaf77ab7615e23822b64a23bdb190e6c3726092a7b19"}
Sep 16 18:13:57.657469 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.657224    2559 scope.go:117] "RemoveContainer" containerID="f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"
Sep 16 18:13:57.659225 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.659188    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" event={"ID":"28d91c42-5e44-48c3-b0d0-224963c0da07","Type":"ContainerStarted","Data":"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"}
Sep 16 18:13:57.659320 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.659252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" event={"ID":"28d91c42-5e44-48c3-b0d0-224963c0da07","Type":"ContainerStarted","Data":"39f67a085b07023c67358941977c05a3c4d177e6b79f725cd51c808f679d4134"}
Sep 16 18:13:57.665809 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.665790    2559 scope.go:117] "RemoveContainer" containerID="f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"
Sep 16 18:13:57.666063 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:13:57.666043    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a\": container with ID starting with f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a not found: ID does not exist" containerID="f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"
Sep 16 18:13:57.666127 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.666073    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a"} err="failed to get container status \"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a\": rpc error: code = NotFound desc = could not find container \"f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a\": container with ID starting with f1ec8af8390ce92829435b73bd5a55ef4835415913b8663874eaa419cdcf497a not found: ID does not exist"
Sep 16 18:13:57.681352 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.681286    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" podStartSLOduration=1.681274616 podStartE2EDuration="1.681274616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:13:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:13:57.680455867 +0000 UTC m=+2960.910237558" watchObservedRunningTime="2026-09-16 18:13:57.681274616 +0000 UTC m=+2960.911056355"
Sep 16 18:13:57.695908 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.695885    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:13:57.697437 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.697420    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-5btlf"]
Sep 16 18:13:57.772162 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:57.772144    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:13:59.296367 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:13:59.296331    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" path="/var/lib/kubelet/pods/3b6925fe-0881-4157-a382-0f10a3d3c61f/volumes"
Sep 16 18:14:06.772284 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:06.772249    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:14:06.784799 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:06.784777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:14:07.720929 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:07.720894    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:14:19.962722 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.962685    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:14:19.963172 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.963059    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerName="main"
Sep 16 18:14:19.963172 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.963070    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerName="main"
Sep 16 18:14:19.963172 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.963141    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b6925fe-0881-4157-a382-0f10a3d3c61f" containerName="main"
Sep 16 18:14:19.965977 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.965955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:19.968811 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.968789    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v2-kserve-self-signed-certs\""
Sep 16 18:14:19.990729 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:19.990705    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:14:20.108807 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.108936 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.108936 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qm6d\" (UniqueName: \"kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.108936 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.109062 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.109062 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.108999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210097 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210215 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210215 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210299 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210248    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210346 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210296    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qm6d\" (UniqueName: \"kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210389 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210477 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210455    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210597 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.210761 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.210741    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.212317 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.212300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.212511 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.212493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.220231 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.220182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qm6d\" (UniqueName: \"kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d\") pod \"owner-v2-kserve-5cf8959f95-swrbh\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.277694 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.277671    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:20.554334 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.554306    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:14:20.556403 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:14:20.556376    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b33422f_84d2_41b6_a3c9_588aea8ca784.slice/crio-9cc7b9ae62fe44f536395c64ad1e25baca502bc472170376fa3437417e65d4cb WatchSource:0}: Error finding container 9cc7b9ae62fe44f536395c64ad1e25baca502bc472170376fa3437417e65d4cb: Status 404 returned error can't find the container with id 9cc7b9ae62fe44f536395c64ad1e25baca502bc472170376fa3437417e65d4cb
Sep 16 18:14:20.771284 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.771252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" event={"ID":"6b33422f-84d2-41b6-a3c9-588aea8ca784","Type":"ContainerStarted","Data":"0c884e993898845ecf55656d882f9ebd95140295896d214a8e76743a1c32df54"}
Sep 16 18:14:20.771460 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.771304    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" event={"ID":"6b33422f-84d2-41b6-a3c9-588aea8ca784","Type":"ContainerStarted","Data":"9cc7b9ae62fe44f536395c64ad1e25baca502bc472170376fa3437417e65d4cb"}
Sep 16 18:14:20.804732 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:20.804649    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" podStartSLOduration=1.804633041 podStartE2EDuration="1.804633041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:14:20.801693477 +0000 UTC m=+2984.031475168" watchObservedRunningTime="2026-09-16 18:14:20.804633041 +0000 UTC m=+2984.034414732"
Sep 16 18:14:21.278166 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:21.278133    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:30.278392 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:30.278353    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:30.290731 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:30.290707    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:30.827793 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:30.827762    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:14:37.463308 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:37.463277    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:14:37.474179 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:14:37.474158    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:15:18.481010 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:18.480978    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:15:33.499684 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.499605    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerName="main" containerID="cri-o://6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2" gracePeriod=45
Sep 16 18:15:33.748888 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.748864    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:15:33.894407 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894594 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894430    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894594 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894467    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894594 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894486    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894594 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894534    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbtdc\" (UniqueName: \"kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894594 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894580    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm\") pod \"28d91c42-5e44-48c3-b0d0-224963c0da07\" (UID: \"28d91c42-5e44-48c3-b0d0-224963c0da07\") "
Sep 16 18:15:33.894872 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894699    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:33.894872 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894799    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home" (OuterVolumeSpecName: "home") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:33.894872 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894808    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache" (OuterVolumeSpecName: "model-cache") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:33.894872 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.894829    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:33.896712 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.896685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm" (OuterVolumeSpecName: "dshm") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:33.896808 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.896706    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc" (OuterVolumeSpecName: "kube-api-access-lbtdc") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "kube-api-access-lbtdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:15:33.896808 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.896735    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28d91c42-5e44-48c3-b0d0-224963c0da07" (UID: "28d91c42-5e44-48c3-b0d0-224963c0da07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:15:33.995372 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.995348    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:33.995372 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.995371    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:33.995509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.995380    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbtdc\" (UniqueName: \"kubernetes.io/projected/28d91c42-5e44-48c3-b0d0-224963c0da07-kube-api-access-lbtdc\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:33.995509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.995390    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d91c42-5e44-48c3-b0d0-224963c0da07-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:33.995509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:33.995398    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d91c42-5e44-48c3-b0d0-224963c0da07-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:34.126979 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.126952    2559 generic.go:356] "Generic (PLEG): container finished" podID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerID="6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2" exitCode=0
Sep 16 18:15:34.127091 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.126999    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"
Sep 16 18:15:34.127091 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.127048    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" event={"ID":"28d91c42-5e44-48c3-b0d0-224963c0da07","Type":"ContainerDied","Data":"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"}
Sep 16 18:15:34.127179 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.127101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" event={"ID":"28d91c42-5e44-48c3-b0d0-224963c0da07","Type":"ContainerDied","Data":"39f67a085b07023c67358941977c05a3c4d177e6b79f725cd51c808f679d4134"}
Sep 16 18:15:34.127179 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.127129    2559 scope.go:117] "RemoveContainer" containerID="6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"
Sep 16 18:15:34.135937 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.135919    2559 scope.go:117] "RemoveContainer" containerID="6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"
Sep 16 18:15:34.136185 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:15:34.136166    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2\": container with ID starting with 6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2 not found: ID does not exist" containerID="6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"
Sep 16 18:15:34.136235 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.136193    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2"} err="failed to get container status \"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2\": rpc error: code = NotFound desc = could not find container \"6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2\": container with ID starting with 6316d4887d0c56745eb6310b2beef3ce7d6aa0083bbf6d111842f323b600c4a2 not found: ID does not exist"
Sep 16 18:15:34.149126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.149079    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:15:34.152509 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.152488    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9"]
Sep 16 18:15:34.710004 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:34.709969    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-l2ck9" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/health\": context deadline exceeded"
Sep 16 18:15:35.295035 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:35.294989    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" path="/var/lib/kubelet/pods/28d91c42-5e44-48c3-b0d0-224963c0da07/volumes"
Sep 16 18:15:42.040182 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:42.040153    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:15:57.060123 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.060072    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" podUID="6b33422f-84d2-41b6-a3c9-588aea8ca784" containerName="main" containerID="cri-o://0c884e993898845ecf55656d882f9ebd95140295896d214a8e76743a1c32df54" gracePeriod=45
Sep 16 18:15:57.237793 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.237766    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b33422f-84d2-41b6-a3c9-588aea8ca784" containerID="0c884e993898845ecf55656d882f9ebd95140295896d214a8e76743a1c32df54" exitCode=0
Sep 16 18:15:57.237915 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.237815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" event={"ID":"6b33422f-84d2-41b6-a3c9-588aea8ca784","Type":"ContainerDied","Data":"0c884e993898845ecf55656d882f9ebd95140295896d214a8e76743a1c32df54"}
Sep 16 18:15:57.300992 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.300968    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:15:57.371551 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371680 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371579    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371680 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371680 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371632    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371680 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qm6d\" (UniqueName: \"kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371889 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371685    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache\") pod \"6b33422f-84d2-41b6-a3c9-588aea8ca784\" (UID: \"6b33422f-84d2-41b6-a3c9-588aea8ca784\") "
Sep 16 18:15:57.371947 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371900    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:57.372004 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371947    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home" (OuterVolumeSpecName: "home") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:57.372074 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.371992    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:57.372074 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.372012    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:57.373656 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.373631    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d" (OuterVolumeSpecName: "kube-api-access-2qm6d") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "kube-api-access-2qm6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:15:57.373756 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.373679    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm" (OuterVolumeSpecName: "dshm") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:15:57.373756 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.373711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b33422f-84d2-41b6-a3c9-588aea8ca784" (UID: "6b33422f-84d2-41b6-a3c9-588aea8ca784"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:15:57.472592 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.472551    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:57.472592 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.472592    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b33422f-84d2-41b6-a3c9-588aea8ca784-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:57.472716 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.472604    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qm6d\" (UniqueName: \"kubernetes.io/projected/6b33422f-84d2-41b6-a3c9-588aea8ca784-kube-api-access-2qm6d\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:57.472716 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.472613    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:57.472716 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:57.472621    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b33422f-84d2-41b6-a3c9-588aea8ca784-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:15:58.243335 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:58.243298    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh" event={"ID":"6b33422f-84d2-41b6-a3c9-588aea8ca784","Type":"ContainerDied","Data":"9cc7b9ae62fe44f536395c64ad1e25baca502bc472170376fa3437417e65d4cb"}
Sep 16 18:15:58.243335 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:58.243334    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"
Sep 16 18:15:58.243871 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:58.243372    2559 scope.go:117] "RemoveContainer" containerID="0c884e993898845ecf55656d882f9ebd95140295896d214a8e76743a1c32df54"
Sep 16 18:15:58.265310 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:58.265283    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:15:58.271333 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:58.271311    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-swrbh"]
Sep 16 18:15:59.295301 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:15:59.295264    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b33422f-84d2-41b6-a3c9-588aea8ca784" path="/var/lib/kubelet/pods/6b33422f-84d2-41b6-a3c9-588aea8ca784/volumes"
Sep 16 18:16:41.390257 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390224    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:16:41.390683 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390597    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerName="main"
Sep 16 18:16:41.390683 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390610    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerName="main"
Sep 16 18:16:41.390683 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390620    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b33422f-84d2-41b6-a3c9-588aea8ca784" containerName="main"
Sep 16 18:16:41.390683 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390625    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b33422f-84d2-41b6-a3c9-588aea8ca784" containerName="main"
Sep 16 18:16:41.390683 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390681    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b33422f-84d2-41b6-a3c9-588aea8ca784" containerName="main"
Sep 16 18:16:41.390869 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.390692    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d91c42-5e44-48c3-b0d0-224963c0da07" containerName="main"
Sep 16 18:16:41.395225 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.395205    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.399646 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.399623    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Sep 16 18:16:41.399782 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.399647    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Sep 16 18:16:41.399782 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.399623    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-h752m\""
Sep 16 18:16:41.399782 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.399668    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Sep 16 18:16:41.399782 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.399656    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-9bp9x\""
Sep 16 18:16:41.407593 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.407534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:16:41.523672 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.523641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzh8d\" (UniqueName: \"kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.523829 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.523707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.625078 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.625043    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzh8d\" (UniqueName: \"kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.625234 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.625110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.627537 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.627516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.635839 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.635814    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzh8d\" (UniqueName: \"kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.706642 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.706553    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:41.977347 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:41.977300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:16:41.979628 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:16:41.979593    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc4a6b5_a450_4944_8fd3_8b99d8012887.slice/crio-b5fca30af7614986a39d7f427b5df46314f8a4fe5c4c051f0a29b0e3574427cd WatchSource:0}: Error finding container b5fca30af7614986a39d7f427b5df46314f8a4fe5c4c051f0a29b0e3574427cd: Status 404 returned error can't find the container with id b5fca30af7614986a39d7f427b5df46314f8a4fe5c4c051f0a29b0e3574427cd
Sep 16 18:16:42.447680 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:42.447644    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" event={"ID":"5dc4a6b5-a450-4944-8fd3-8b99d8012887","Type":"ContainerStarted","Data":"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"}
Sep 16 18:16:42.448051 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:42.447699    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" event={"ID":"5dc4a6b5-a450-4944-8fd3-8b99d8012887","Type":"ContainerStarted","Data":"b5fca30af7614986a39d7f427b5df46314f8a4fe5c4c051f0a29b0e3574427cd"}
Sep 16 18:16:42.448051 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:42.447761    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:16:42.465235 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:16:42.465190    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podStartSLOduration=1.465172471 podStartE2EDuration="1.465172471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:42.464357181 +0000 UTC m=+3125.694138873" watchObservedRunningTime="2026-09-16 18:16:42.465172471 +0000 UTC m=+3125.694954162"
Sep 16 18:17:13.453435 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:13.453408    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:17:14.911140 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:14.911104    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:17:23.453953 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:23.453914    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:17:29.931679 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:29.931621    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main" containerID="cri-o://e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77" gracePeriod=45
Sep 16 18:17:33.454475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:33.454427    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:17:43.454357 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:43.454317    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:17:43.454904 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:43.454430    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:17:53.453877 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:17:53.453839    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:18:00.180737 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.180714    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:18:00.243337 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.243278    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzh8d\" (UniqueName: \"kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d\") pod \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") "
Sep 16 18:18:00.243337 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.243323    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs\") pod \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\" (UID: \"5dc4a6b5-a450-4944-8fd3-8b99d8012887\") "
Sep 16 18:18:00.245388 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.245363    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5dc4a6b5-a450-4944-8fd3-8b99d8012887" (UID: "5dc4a6b5-a450-4944-8fd3-8b99d8012887"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:18:00.245497 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.245429    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d" (OuterVolumeSpecName: "kube-api-access-jzh8d") pod "5dc4a6b5-a450-4944-8fd3-8b99d8012887" (UID: "5dc4a6b5-a450-4944-8fd3-8b99d8012887"). InnerVolumeSpecName "kube-api-access-jzh8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:18:00.344776 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.344744    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzh8d\" (UniqueName: \"kubernetes.io/projected/5dc4a6b5-a450-4944-8fd3-8b99d8012887-kube-api-access-jzh8d\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:18:00.344776 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.344771    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc4a6b5-a450-4944-8fd3-8b99d8012887-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:18:00.808916 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.808878    2559 generic.go:356] "Generic (PLEG): container finished" podID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerID="e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77" exitCode=0
Sep 16 18:18:00.809114 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.808924    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"
Sep 16 18:18:00.809114 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.808977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" event={"ID":"5dc4a6b5-a450-4944-8fd3-8b99d8012887","Type":"ContainerDied","Data":"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"}
Sep 16 18:18:00.809114 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.809039    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq" event={"ID":"5dc4a6b5-a450-4944-8fd3-8b99d8012887","Type":"ContainerDied","Data":"b5fca30af7614986a39d7f427b5df46314f8a4fe5c4c051f0a29b0e3574427cd"}
Sep 16 18:18:00.809114 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.809071    2559 scope.go:117] "RemoveContainer" containerID="e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"
Sep 16 18:18:00.817772 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.817753    2559 scope.go:117] "RemoveContainer" containerID="e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"
Sep 16 18:18:00.818044 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:18:00.818026    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77\": container with ID starting with e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77 not found: ID does not exist" containerID="e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"
Sep 16 18:18:00.818092 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.818054    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77"} err="failed to get container status \"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77\": rpc error: code = NotFound desc = could not find container \"e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77\": container with ID starting with e44de71d0645379a4b357f8431a5710814136bdf9450e94f7a3bb130035b1c77 not found: ID does not exist"
Sep 16 18:18:00.831252 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.831228    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:18:00.836063 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:00.836040    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqb2vq"]
Sep 16 18:18:01.295134 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:01.295106    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" path="/var/lib/kubelet/pods/5dc4a6b5-a450-4944-8fd3-8b99d8012887/volumes"
Sep 16 18:18:31.609275 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.609202    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:18:31.609725 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.609528    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main"
Sep 16 18:18:31.609725 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.609540    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main"
Sep 16 18:18:31.609725 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.609646    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc4a6b5-a450-4944-8fd3-8b99d8012887" containerName="main"
Sep 16 18:18:31.612495 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.612474    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.615433 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.615411    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Sep 16 18:18:31.616736 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.616720    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-kt29k\""
Sep 16 18:18:31.616797 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.616742    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Sep 16 18:18:31.616890 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.616872    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Sep 16 18:18:31.617099 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.617085    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-rh68r\""
Sep 16 18:18:31.626484 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.626465    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:18:31.692421 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.692400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.692510 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.692443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j4gk\" (UniqueName: \"kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.793076 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.793051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.793202 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.793098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j4gk\" (UniqueName: \"kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.795483 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.795456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.802087 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.802066    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j4gk\" (UniqueName: \"kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:31.922778 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:31.922717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:32.195966 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:32.195873    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:18:32.198190 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:18:32.198163    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52175cb1_e1b2_4b58_b415_2b557e2b25dd.slice/crio-c1fe988df3a5c7c25e5fc712fe4320e38879c2a5973d6df49de4040dbc3eacae WatchSource:0}: Error finding container c1fe988df3a5c7c25e5fc712fe4320e38879c2a5973d6df49de4040dbc3eacae: Status 404 returned error can't find the container with id c1fe988df3a5c7c25e5fc712fe4320e38879c2a5973d6df49de4040dbc3eacae
Sep 16 18:18:32.958265 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:32.958221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" event={"ID":"52175cb1-e1b2-4b58-b415-2b557e2b25dd","Type":"ContainerStarted","Data":"f49c14628276d49be571fbff8ddf4c09fa6b7bac621acdba4ea1931807dc00b5"}
Sep 16 18:18:32.958703 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:32.958291    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" event={"ID":"52175cb1-e1b2-4b58-b415-2b557e2b25dd","Type":"ContainerStarted","Data":"c1fe988df3a5c7c25e5fc712fe4320e38879c2a5973d6df49de4040dbc3eacae"}
Sep 16 18:18:32.958703 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:32.958422    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:18:32.979063 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:18:32.979020    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podStartSLOduration=1.97900532 podStartE2EDuration="1.97900532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:32.977863842 +0000 UTC m=+3236.207645533" watchObservedRunningTime="2026-09-16 18:18:32.97900532 +0000 UTC m=+3236.208787011"
Sep 16 18:19:03.964927 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:19:03.964899    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:19:37.494771 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:19:37.494739    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:19:37.508318 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:19:37.508298    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:20:36.568405 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:20:36.568373    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:20:43.965102 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:20:43.965058    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:20:51.587946 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:20:51.587904    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main" containerID="cri-o://f49c14628276d49be571fbff8ddf4c09fa6b7bac621acdba4ea1931807dc00b5" gracePeriod=45
Sep 16 18:20:53.965025 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:20:53.964990    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:21:03.965291 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:03.965256    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:21:03.965746 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:03.965353    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:21:13.964974 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:13.964937    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:21:21.745478 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:21.745447    2559 generic.go:356] "Generic (PLEG): container finished" podID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerID="f49c14628276d49be571fbff8ddf4c09fa6b7bac621acdba4ea1931807dc00b5" exitCode=0
Sep 16 18:21:21.745856 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:21.745517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" event={"ID":"52175cb1-e1b2-4b58-b415-2b557e2b25dd","Type":"ContainerDied","Data":"f49c14628276d49be571fbff8ddf4c09fa6b7bac621acdba4ea1931807dc00b5"}
Sep 16 18:21:21.830133 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:21.830110    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:21:22.010663 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.010584    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs\") pod \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") "
Sep 16 18:21:22.010663 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.010623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j4gk\" (UniqueName: \"kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk\") pod \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\" (UID: \"52175cb1-e1b2-4b58-b415-2b557e2b25dd\") "
Sep 16 18:21:22.012775 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.012751    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52175cb1-e1b2-4b58-b415-2b557e2b25dd" (UID: "52175cb1-e1b2-4b58-b415-2b557e2b25dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:21:22.012901 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.012827    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk" (OuterVolumeSpecName: "kube-api-access-7j4gk") pod "52175cb1-e1b2-4b58-b415-2b557e2b25dd" (UID: "52175cb1-e1b2-4b58-b415-2b557e2b25dd"). InnerVolumeSpecName "kube-api-access-7j4gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:21:22.111301 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.111277    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52175cb1-e1b2-4b58-b415-2b557e2b25dd-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:21:22.111301 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.111299    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j4gk\" (UniqueName: \"kubernetes.io/projected/52175cb1-e1b2-4b58-b415-2b557e2b25dd-kube-api-access-7j4gk\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:21:22.751360 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.751323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl" event={"ID":"52175cb1-e1b2-4b58-b415-2b557e2b25dd","Type":"ContainerDied","Data":"c1fe988df3a5c7c25e5fc712fe4320e38879c2a5973d6df49de4040dbc3eacae"}
Sep 16 18:21:22.751787 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.751380    2559 scope.go:117] "RemoveContainer" containerID="f49c14628276d49be571fbff8ddf4c09fa6b7bac621acdba4ea1931807dc00b5"
Sep 16 18:21:22.751787 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.751334    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"
Sep 16 18:21:22.775431 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.775409    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:21:22.781495 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:22.781467    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schehprgl"]
Sep 16 18:21:23.295581 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:21:23.295529    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" path="/var/lib/kubelet/pods/52175cb1-e1b2-4b58-b415-2b557e2b25dd/volumes"
Sep 16 18:22:04.188671 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.188636    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:22:04.189075 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.188971    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main"
Sep 16 18:22:04.189075 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.188982    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main"
Sep 16 18:22:04.189075 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.189070    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="52175cb1-e1b2-4b58-b415-2b557e2b25dd" containerName="main"
Sep 16 18:22:04.192213 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.192192    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.197081 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.197060    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-z5dqf\""
Sep 16 18:22:04.197081 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.197074    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"openshift-service-ca.crt\""
Sep 16 18:22:04.197257 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.197107    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"kube-root-ca.crt\""
Sep 16 18:22:04.197257 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.197111    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"default-dockercfg-nv9rh\""
Sep 16 18:22:04.197257 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.197128    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Sep 16 18:22:04.204058 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.204039    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:22:04.358276 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.358243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.358276 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.358277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxfsg\" (UniqueName: \"kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.459498 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.459411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.459498 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.459446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxfsg\" (UniqueName: \"kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.462040 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.462017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.471172 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.471144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxfsg\" (UniqueName: \"kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.503099 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.503074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.779784 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.779762    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:22:04.781719 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:22:04.781688    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b64d30_78c7_4246_9769_59fb8f81b306.slice/crio-c5a138c31a9a8ffa2e0f3c306d4f9348dfe97bafc0bd02f3c9ae3a5ce6bac5f0 WatchSource:0}: Error finding container c5a138c31a9a8ffa2e0f3c306d4f9348dfe97bafc0bd02f3c9ae3a5ce6bac5f0: Status 404 returned error can't find the container with id c5a138c31a9a8ffa2e0f3c306d4f9348dfe97bafc0bd02f3c9ae3a5ce6bac5f0
Sep 16 18:22:04.953851 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.953819    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" event={"ID":"f9b64d30-78c7-4246-9769-59fb8f81b306","Type":"ContainerStarted","Data":"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"}
Sep 16 18:22:04.954015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.953872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" event={"ID":"f9b64d30-78c7-4246-9769-59fb8f81b306","Type":"ContainerStarted","Data":"c5a138c31a9a8ffa2e0f3c306d4f9348dfe97bafc0bd02f3c9ae3a5ce6bac5f0"}
Sep 16 18:22:04.954015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.953893    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:22:04.973665 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:04.973623    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podStartSLOduration=0.973607878 podStartE2EDuration="973.607878ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:22:04.972637443 +0000 UTC m=+3448.202419135" watchObservedRunningTime="2026-09-16 18:22:04.973607878 +0000 UTC m=+3448.203389584"
Sep 16 18:22:35.959747 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:22:35.959713    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:23:35.960588 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:35.960537    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:23:35.994522 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:35.994489    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:23:45.960150 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:45.960107    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:23:51.013385 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:51.013313    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" containerID="cri-o://24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d" gracePeriod=45
Sep 16 18:23:55.960476 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:55.960439    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:23:55.960876 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:23:55.960539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:24:05.960743 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:05.960707    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:24:15.960065 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:15.960028    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:24:21.255619 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.255596    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:24:21.363692 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.363668    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs\") pod \"f9b64d30-78c7-4246-9769-59fb8f81b306\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") "
Sep 16 18:24:21.363833 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.363697    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxfsg\" (UniqueName: \"kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg\") pod \"f9b64d30-78c7-4246-9769-59fb8f81b306\" (UID: \"f9b64d30-78c7-4246-9769-59fb8f81b306\") "
Sep 16 18:24:21.365762 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.365738    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9b64d30-78c7-4246-9769-59fb8f81b306" (UID: "f9b64d30-78c7-4246-9769-59fb8f81b306"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:24:21.365857 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.365840    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg" (OuterVolumeSpecName: "kube-api-access-jxfsg") pod "f9b64d30-78c7-4246-9769-59fb8f81b306" (UID: "f9b64d30-78c7-4246-9769-59fb8f81b306"). InnerVolumeSpecName "kube-api-access-jxfsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:24:21.464510 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.464491    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b64d30-78c7-4246-9769-59fb8f81b306-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:24:21.464510 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.464509    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxfsg\" (UniqueName: \"kubernetes.io/projected/f9b64d30-78c7-4246-9769-59fb8f81b306-kube-api-access-jxfsg\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:24:21.564159 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.564134    2559 generic.go:356] "Generic (PLEG): container finished" podID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerID="24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d" exitCode=0
Sep 16 18:24:21.564259 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.564173    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"
Sep 16 18:24:21.564259 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.564225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" event={"ID":"f9b64d30-78c7-4246-9769-59fb8f81b306","Type":"ContainerDied","Data":"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"}
Sep 16 18:24:21.564332 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.564276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx" event={"ID":"f9b64d30-78c7-4246-9769-59fb8f81b306","Type":"ContainerDied","Data":"c5a138c31a9a8ffa2e0f3c306d4f9348dfe97bafc0bd02f3c9ae3a5ce6bac5f0"}
Sep 16 18:24:21.564332 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.564303    2559 scope.go:117] "RemoveContainer" containerID="24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"
Sep 16 18:24:21.573970 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.573955    2559 scope.go:117] "RemoveContainer" containerID="24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"
Sep 16 18:24:21.574212 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:24:21.574190    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d\": container with ID starting with 24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d not found: ID does not exist" containerID="24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"
Sep 16 18:24:21.574272 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.574219    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d"} err="failed to get container status \"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d\": rpc error: code = NotFound desc = could not find container \"24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d\": container with ID starting with 24e4aab89ecab326be090a139345333b4a4bc2f5f6ae2bfb75b52bc15176ec4d not found: ID does not exist"
Sep 16 18:24:21.588459 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.588433    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:24:21.592605 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:21.592587    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche5tjtx"]
Sep 16 18:24:23.300575 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:23.296805    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" path="/var/lib/kubelet/pods/f9b64d30-78c7-4246-9769-59fb8f81b306/volumes"
Sep 16 18:24:37.527954 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:37.527922    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:24:37.538753 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:37.538731    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:24:48.291753 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.291698    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:24:48.292222 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.292188    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main"
Sep 16 18:24:48.292222 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.292209    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main"
Sep 16 18:24:48.292345 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.292298    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b64d30-78c7-4246-9769-59fb8f81b306" containerName="main"
Sep 16 18:24:48.296411 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.296391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.300723 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.300701    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Sep 16 18:24:48.300850 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.300726    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-hrbmp\""
Sep 16 18:24:48.300850 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.300814    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Sep 16 18:24:48.300850 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.300730    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-q5hhl\""
Sep 16 18:24:48.301025 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.300953    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Sep 16 18:24:48.306934 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.306912    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:24:48.362373 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.362346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.362824 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.362794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvvt\" (UniqueName: \"kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.463679 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.463643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvvt\" (UniqueName: \"kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.463809 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.463707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.466138 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.466121    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.472853 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.472821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvvt\" (UniqueName: \"kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.608537 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.608507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:48.881139 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:48.881037    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:24:48.883399 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:24:48.883371    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4e36a4_ce1c_436e_97c6_dea8709d2fbd.slice/crio-407e36705ef239057e9e08c407ca2a40c100224e420da7556dba4163f290979d WatchSource:0}: Error finding container 407e36705ef239057e9e08c407ca2a40c100224e420da7556dba4163f290979d: Status 404 returned error can't find the container with id 407e36705ef239057e9e08c407ca2a40c100224e420da7556dba4163f290979d
Sep 16 18:24:49.695691 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:49.695656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" event={"ID":"af4e36a4-ce1c-436e-97c6-dea8709d2fbd","Type":"ContainerStarted","Data":"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"}
Sep 16 18:24:49.696053 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:49.695710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" event={"ID":"af4e36a4-ce1c-436e-97c6-dea8709d2fbd","Type":"ContainerStarted","Data":"407e36705ef239057e9e08c407ca2a40c100224e420da7556dba4163f290979d"}
Sep 16 18:24:49.696053 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:49.695742    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:24:49.721687 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:24:49.721640    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podStartSLOduration=1.721628381 podStartE2EDuration="1.721628381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:24:49.719349465 +0000 UTC m=+3612.949131157" watchObservedRunningTime="2026-09-16 18:24:49.721628381 +0000 UTC m=+3612.951410072"
Sep 16 18:25:20.702678 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:20.702648    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:25:30.311717 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.311684    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:25:30.320538 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.320511    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.325084 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.325058    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Sep 16 18:25:30.325224 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.325147    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-pskff\""
Sep 16 18:25:30.325331 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.325303    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Sep 16 18:25:30.325410 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.325362    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Sep 16 18:25:30.325531 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.325514    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-l868g\""
Sep 16 18:25:30.327329 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.327303    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:25:30.491907 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.491875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29wp\" (UniqueName: \"kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.492057 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.491926    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.593231 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.593188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29wp\" (UniqueName: \"kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.593364 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.593244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.595639 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.595618    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.601443 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.601422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29wp\" (UniqueName: \"kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.634023 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.634000    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:30.911080 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:30.910900    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:25:30.913342 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:25:30.913312    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd68491_b26e_4810_bd32_622528cddc70.slice/crio-db3ba7dd5b995115f3e8b62ad8d1a3536c672a373893396757e4a5b2b725cb55 WatchSource:0}: Error finding container db3ba7dd5b995115f3e8b62ad8d1a3536c672a373893396757e4a5b2b725cb55: Status 404 returned error can't find the container with id db3ba7dd5b995115f3e8b62ad8d1a3536c672a373893396757e4a5b2b725cb55
Sep 16 18:25:31.890374 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:31.890338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" event={"ID":"4dd68491-b26e-4810-bd32-622528cddc70","Type":"ContainerStarted","Data":"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"}
Sep 16 18:25:31.890769 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:31.890394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" event={"ID":"4dd68491-b26e-4810-bd32-622528cddc70","Type":"ContainerStarted","Data":"db3ba7dd5b995115f3e8b62ad8d1a3536c672a373893396757e4a5b2b725cb55"}
Sep 16 18:25:31.890769 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:31.890441    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:25:31.912496 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:25:31.912449    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podStartSLOduration=1.9124353219999999 podStartE2EDuration="1.912435322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:25:31.911271229 +0000 UTC m=+3655.141052919" watchObservedRunningTime="2026-09-16 18:25:31.912435322 +0000 UTC m=+3655.142217013"
Sep 16 18:26:02.896800 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:26:02.896727    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:26:53.383807 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:26:53.383770    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:27:00.702303 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:00.702266    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:08.403471 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:08.403425    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main" containerID="cri-o://75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551" gracePeriod=45
Sep 16 18:27:10.701830 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:10.701795    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:20.702519 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:20.702486    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:20.702926 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:20.702604    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:27:30.701873 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:30.701792    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:32.343905 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:32.343872    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:27:32.897241 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:32.897204    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:38.651024 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.650996    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:27:38.763303 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.763232    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbvvt\" (UniqueName: \"kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt\") pod \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") "
Sep 16 18:27:38.763303 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.763272    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs\") pod \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\" (UID: \"af4e36a4-ce1c-436e-97c6-dea8709d2fbd\") "
Sep 16 18:27:38.765436 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.765404    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af4e36a4-ce1c-436e-97c6-dea8709d2fbd" (UID: "af4e36a4-ce1c-436e-97c6-dea8709d2fbd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:27:38.765536 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.765445    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt" (OuterVolumeSpecName: "kube-api-access-rbvvt") pod "af4e36a4-ce1c-436e-97c6-dea8709d2fbd" (UID: "af4e36a4-ce1c-436e-97c6-dea8709d2fbd"). InnerVolumeSpecName "kube-api-access-rbvvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:27:38.863878 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.863856    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbvvt\" (UniqueName: \"kubernetes.io/projected/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-kube-api-access-rbvvt\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:27:38.863878 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:38.863877    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4e36a4-ce1c-436e-97c6-dea8709d2fbd-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:27:39.472864 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.472834    2559 generic.go:356] "Generic (PLEG): container finished" podID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerID="75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551" exitCode=0
Sep 16 18:27:39.473015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.472879    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"
Sep 16 18:27:39.473015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.472891    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" event={"ID":"af4e36a4-ce1c-436e-97c6-dea8709d2fbd","Type":"ContainerDied","Data":"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"}
Sep 16 18:27:39.473015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.472938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq" event={"ID":"af4e36a4-ce1c-436e-97c6-dea8709d2fbd","Type":"ContainerDied","Data":"407e36705ef239057e9e08c407ca2a40c100224e420da7556dba4163f290979d"}
Sep 16 18:27:39.473015 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.472959    2559 scope.go:117] "RemoveContainer" containerID="75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"
Sep 16 18:27:39.481521 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.481504    2559 scope.go:117] "RemoveContainer" containerID="75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"
Sep 16 18:27:39.481815 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:27:39.481793    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551\": container with ID starting with 75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551 not found: ID does not exist" containerID="75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"
Sep 16 18:27:39.481870 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.481825    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551"} err="failed to get container status \"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551\": rpc error: code = NotFound desc = could not find container \"75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551\": container with ID starting with 75d4edf72ead3bd3e43450bb47f54ec8507bfb5c84f309d38ae0b88c84554551 not found: ID does not exist"
Sep 16 18:27:39.489928 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.489906    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:27:39.496153 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:39.496131    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58xtsvq"]
Sep 16 18:27:41.295309 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:41.295276    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" path="/var/lib/kubelet/pods/af4e36a4-ce1c-436e-97c6-dea8709d2fbd/volumes"
Sep 16 18:27:42.896503 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:42.896465    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:47.364417 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:47.364379    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" containerID="cri-o://7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb" gracePeriod=45
Sep 16 18:27:52.897453 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:52.897416    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:27:52.897827 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:27:52.897513    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:28:02.896622 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:02.896581    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:28:12.897318 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:12.897278    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:28:17.618676 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.618650    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:28:17.655580 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.655494    2559 generic.go:356] "Generic (PLEG): container finished" podID="4dd68491-b26e-4810-bd32-622528cddc70" containerID="7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb" exitCode=0
Sep 16 18:28:17.655580 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.655550    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"
Sep 16 18:28:17.655746 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.655585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" event={"ID":"4dd68491-b26e-4810-bd32-622528cddc70","Type":"ContainerDied","Data":"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"}
Sep 16 18:28:17.655746 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.655646    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f" event={"ID":"4dd68491-b26e-4810-bd32-622528cddc70","Type":"ContainerDied","Data":"db3ba7dd5b995115f3e8b62ad8d1a3536c672a373893396757e4a5b2b725cb55"}
Sep 16 18:28:17.655746 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.655674    2559 scope.go:117] "RemoveContainer" containerID="7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"
Sep 16 18:28:17.664806 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.664781    2559 scope.go:117] "RemoveContainer" containerID="7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"
Sep 16 18:28:17.665095 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:28:17.665072    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb\": container with ID starting with 7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb not found: ID does not exist" containerID="7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"
Sep 16 18:28:17.665163 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.665106    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb"} err="failed to get container status \"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb\": rpc error: code = NotFound desc = could not find container \"7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb\": container with ID starting with 7019c52bc208611490489ccc1b73b76a3fb3c3cd662e89353c7cec4b746760bb not found: ID does not exist"
Sep 16 18:28:17.672359 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.672340    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs\") pod \"4dd68491-b26e-4810-bd32-622528cddc70\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") "
Sep 16 18:28:17.672459 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.672448    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n29wp\" (UniqueName: \"kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp\") pod \"4dd68491-b26e-4810-bd32-622528cddc70\" (UID: \"4dd68491-b26e-4810-bd32-622528cddc70\") "
Sep 16 18:28:17.674442 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.674413    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4dd68491-b26e-4810-bd32-622528cddc70" (UID: "4dd68491-b26e-4810-bd32-622528cddc70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:28:17.674520 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.674468    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp" (OuterVolumeSpecName: "kube-api-access-n29wp") pod "4dd68491-b26e-4810-bd32-622528cddc70" (UID: "4dd68491-b26e-4810-bd32-622528cddc70"). InnerVolumeSpecName "kube-api-access-n29wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:28:17.773126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.773103    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n29wp\" (UniqueName: \"kubernetes.io/projected/4dd68491-b26e-4810-bd32-622528cddc70-kube-api-access-n29wp\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:28:17.773126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.773123    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd68491-b26e-4810-bd32-622528cddc70-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:28:17.978838 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.978812    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:28:17.982670 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:17.982648    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-lfq8f"]
Sep 16 18:28:19.295687 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:19.295650    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd68491-b26e-4810-bd32-622528cddc70" path="/var/lib/kubelet/pods/4dd68491-b26e-4810-bd32-622528cddc70/volumes"
Sep 16 18:28:21.060943 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.060911    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:28:21.061346 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061314    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main"
Sep 16 18:28:21.061346 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061329    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main"
Sep 16 18:28:21.061346 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061342    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main"
Sep 16 18:28:21.061346 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061348    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main"
Sep 16 18:28:21.061496 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061409    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4e36a4-ce1c-436e-97c6-dea8709d2fbd" containerName="main"
Sep 16 18:28:21.061496 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.061417    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd68491-b26e-4810-bd32-622528cddc70" containerName="main"
Sep 16 18:28:21.065988 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.065965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.069905 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.069882    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Sep 16 18:28:21.070105 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.070091    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-h4vf4\""
Sep 16 18:28:21.070195 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.070144    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Sep 16 18:28:21.070288 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.070192    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Sep 16 18:28:21.070288 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.070262    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-l868g\""
Sep 16 18:28:21.076093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.076065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:28:21.097126 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.097100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vtmx\" (UniqueName: \"kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.097219 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.097132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.197737 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.197709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vtmx\" (UniqueName: \"kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.197837 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.197742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.200191 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.200171    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.207707 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.207686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vtmx\" (UniqueName: \"kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.381629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.381606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:21.655119 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.655095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:28:21.657119 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:28:21.657090    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da21eec_4d76_4804_bd27_746c8c08b71e.slice/crio-f7d825d95b0f14a25531b49bfebb2ef6212a7d44b6546c30cd839df94ec4bf0d WatchSource:0}: Error finding container f7d825d95b0f14a25531b49bfebb2ef6212a7d44b6546c30cd839df94ec4bf0d: Status 404 returned error can't find the container with id f7d825d95b0f14a25531b49bfebb2ef6212a7d44b6546c30cd839df94ec4bf0d
Sep 16 18:28:21.676274 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:21.676248    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" event={"ID":"0da21eec-4d76-4804-bd27-746c8c08b71e","Type":"ContainerStarted","Data":"f7d825d95b0f14a25531b49bfebb2ef6212a7d44b6546c30cd839df94ec4bf0d"}
Sep 16 18:28:22.682607 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:22.682540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" event={"ID":"0da21eec-4d76-4804-bd27-746c8c08b71e","Type":"ContainerStarted","Data":"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"}
Sep 16 18:28:22.682986 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:22.682721    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:28:22.702185 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:22.702137    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podStartSLOduration=1.702123733 podStartE2EDuration="1.702123733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:22.701738715 +0000 UTC m=+3825.931520431" watchObservedRunningTime="2026-09-16 18:28:22.702123733 +0000 UTC m=+3825.931905424"
Sep 16 18:28:53.688734 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:28:53.688703    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:29:37.558027 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:29:37.558002    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:29:37.575008 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:29:37.574987    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:30:52.891873 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:30:52.891839    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:30:53.689243 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:30:53.689205    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:31:03.688905 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:03.688862    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:31:07.910692 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:07.910633    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" containerID="cri-o://b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6" gracePeriod=45
Sep 16 18:31:13.688978 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:13.688934    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:31:13.689354 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:13.689037    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:31:23.689173 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:23.689125    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:31:33.689316 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:33.689275    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:31:38.153484 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.153460    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:31:38.218381 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.218317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs\") pod \"0da21eec-4d76-4804-bd27-746c8c08b71e\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") "
Sep 16 18:31:38.218381 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.218373    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vtmx\" (UniqueName: \"kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx\") pod \"0da21eec-4d76-4804-bd27-746c8c08b71e\" (UID: \"0da21eec-4d76-4804-bd27-746c8c08b71e\") "
Sep 16 18:31:38.220443 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.220409    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0da21eec-4d76-4804-bd27-746c8c08b71e" (UID: "0da21eec-4d76-4804-bd27-746c8c08b71e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:31:38.220544 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.220419    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx" (OuterVolumeSpecName: "kube-api-access-6vtmx") pod "0da21eec-4d76-4804-bd27-746c8c08b71e" (UID: "0da21eec-4d76-4804-bd27-746c8c08b71e"). InnerVolumeSpecName "kube-api-access-6vtmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:31:38.318994 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.318963    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da21eec-4d76-4804-bd27-746c8c08b71e-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:31:38.319093 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.318994    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vtmx\" (UniqueName: \"kubernetes.io/projected/0da21eec-4d76-4804-bd27-746c8c08b71e-kube-api-access-6vtmx\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:31:38.586426 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.586396    2559 generic.go:356] "Generic (PLEG): container finished" podID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerID="b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6" exitCode=0
Sep 16 18:31:38.586554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.586442    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"
Sep 16 18:31:38.586554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.586481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" event={"ID":"0da21eec-4d76-4804-bd27-746c8c08b71e","Type":"ContainerDied","Data":"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"}
Sep 16 18:31:38.586554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.586543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn" event={"ID":"0da21eec-4d76-4804-bd27-746c8c08b71e","Type":"ContainerDied","Data":"f7d825d95b0f14a25531b49bfebb2ef6212a7d44b6546c30cd839df94ec4bf0d"}
Sep 16 18:31:38.586696 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.586582    2559 scope.go:117] "RemoveContainer" containerID="b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"
Sep 16 18:31:38.596572 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.596539    2559 scope.go:117] "RemoveContainer" containerID="b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"
Sep 16 18:31:38.596838 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:31:38.596820    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6\": container with ID starting with b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6 not found: ID does not exist" containerID="b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"
Sep 16 18:31:38.596888 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.596844    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6"} err="failed to get container status \"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6\": rpc error: code = NotFound desc = could not find container \"b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6\": container with ID starting with b850ec64cdb3060e0aed76685dab3f24deadc477c83ddebdabc01c55151d71c6 not found: ID does not exist"
Sep 16 18:31:38.609456 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.609433    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:31:38.613147 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:38.613129    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-8k7cn"]
Sep 16 18:31:39.296189 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:39.296161    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" path="/var/lib/kubelet/pods/0da21eec-4d76-4804-bd27-746c8c08b71e/volumes"
Sep 16 18:31:44.539153 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.539114    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:31:44.539650 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.539631    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main"
Sep 16 18:31:44.539741 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.539653    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main"
Sep 16 18:31:44.539802 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.539775    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0da21eec-4d76-4804-bd27-746c8c08b71e" containerName="main"
Sep 16 18:31:44.544494 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.544465    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.548595 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.548545    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-sltj4\""
Sep 16 18:31:44.548724 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.548610    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Sep 16 18:31:44.548724 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.548551    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Sep 16 18:31:44.548724 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.548637    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Sep 16 18:31:44.548724 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.548708    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-zrgww\""
Sep 16 18:31:44.553859 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.553838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:31:44.558764 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.558738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsxmj\" (UniqueName: \"kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.558843 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.558816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.659448 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.659417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsxmj\" (UniqueName: \"kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.659614 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.659483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.661915 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.661895    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.668314 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.668290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsxmj\" (UniqueName: \"kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:44.856092 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:44.856063    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:45.143984 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:45.143915    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:31:45.148127 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:31:45.148101    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8929d9b_ced3_4422_a087_2afd88402f97.slice/crio-fab64ad4dd838817edfb8671ef2ea1369866ec6763c48a1032b9c3fe498c0438 WatchSource:0}: Error finding container fab64ad4dd838817edfb8671ef2ea1369866ec6763c48a1032b9c3fe498c0438: Status 404 returned error can't find the container with id fab64ad4dd838817edfb8671ef2ea1369866ec6763c48a1032b9c3fe498c0438
Sep 16 18:31:45.622271 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:45.622232    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" event={"ID":"b8929d9b-ced3-4422-a087-2afd88402f97","Type":"ContainerStarted","Data":"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"}
Sep 16 18:31:45.622689 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:45.622302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" event={"ID":"b8929d9b-ced3-4422-a087-2afd88402f97","Type":"ContainerStarted","Data":"fab64ad4dd838817edfb8671ef2ea1369866ec6763c48a1032b9c3fe498c0438"}
Sep 16 18:31:45.622689 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:45.622331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:31:45.642253 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:31:45.642177    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podStartSLOduration=1.642156868 podStartE2EDuration="1.642156868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:31:45.641684106 +0000 UTC m=+4028.871465798" watchObservedRunningTime="2026-09-16 18:31:45.642156868 +0000 UTC m=+4028.871938569"
Sep 16 18:32:16.629132 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:32:16.629097    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:34:14.636972 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:14.636927    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:34:16.629651 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:16.629608    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:34:26.629638 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:26.629596    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:34:29.657751 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:29.657678    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" containerID="cri-o://873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246" gracePeriod=45
Sep 16 18:34:36.629014 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:36.628973    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:34:36.629429 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:36.629117    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:34:37.587612 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:37.587580    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:34:37.604999 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:37.604971    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:34:46.629141 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:46.629103    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:34:56.629506 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:56.629459    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:34:59.917855 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:34:59.917775    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:35:00.014350 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.014304    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsxmj\" (UniqueName: \"kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj\") pod \"b8929d9b-ced3-4422-a087-2afd88402f97\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") "
Sep 16 18:35:00.014350 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.014353    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs\") pod \"b8929d9b-ced3-4422-a087-2afd88402f97\" (UID: \"b8929d9b-ced3-4422-a087-2afd88402f97\") "
Sep 16 18:35:00.016653 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.016616    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8929d9b-ced3-4422-a087-2afd88402f97" (UID: "b8929d9b-ced3-4422-a087-2afd88402f97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:35:00.016653 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.016631    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj" (OuterVolumeSpecName: "kube-api-access-nsxmj") pod "b8929d9b-ced3-4422-a087-2afd88402f97" (UID: "b8929d9b-ced3-4422-a087-2afd88402f97"). InnerVolumeSpecName "kube-api-access-nsxmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:00.115768 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.115715    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsxmj\" (UniqueName: \"kubernetes.io/projected/b8929d9b-ced3-4422-a087-2afd88402f97-kube-api-access-nsxmj\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.115768 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.115762    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8929d9b-ced3-4422-a087-2afd88402f97-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.549542 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.549501    2559 generic.go:356] "Generic (PLEG): container finished" podID="b8929d9b-ced3-4422-a087-2afd88402f97" containerID="873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246" exitCode=0
Sep 16 18:35:00.549744 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.549571    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"
Sep 16 18:35:00.549744 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.549583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" event={"ID":"b8929d9b-ced3-4422-a087-2afd88402f97","Type":"ContainerDied","Data":"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"}
Sep 16 18:35:00.549744 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.549642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f" event={"ID":"b8929d9b-ced3-4422-a087-2afd88402f97","Type":"ContainerDied","Data":"fab64ad4dd838817edfb8671ef2ea1369866ec6763c48a1032b9c3fe498c0438"}
Sep 16 18:35:00.549744 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.549673    2559 scope.go:117] "RemoveContainer" containerID="873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"
Sep 16 18:35:00.559226 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.559204    2559 scope.go:117] "RemoveContainer" containerID="873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"
Sep 16 18:35:00.559535 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:35:00.559518    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246\": container with ID starting with 873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246 not found: ID does not exist" containerID="873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"
Sep 16 18:35:00.559611 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.559542    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246"} err="failed to get container status \"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246\": rpc error: code = NotFound desc = could not find container \"873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246\": container with ID starting with 873d3ad9759cdb2ae97fe245b7d8770485529008df98641b4f529c737eee3246 not found: ID does not exist"
Sep 16 18:35:00.572763 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.572728    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:35:00.578995 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:00.578963    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schebtr2f"]
Sep 16 18:35:01.295928 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:01.295894    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" path="/var/lib/kubelet/pods/b8929d9b-ced3-4422-a087-2afd88402f97/volumes"
Sep 16 18:35:08.354868 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.354826    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:35:08.355238 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.355201    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main"
Sep 16 18:35:08.355238 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.355213    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main"
Sep 16 18:35:08.355315 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.355291    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8929d9b-ced3-4422-a087-2afd88402f97" containerName="main"
Sep 16 18:35:08.359417 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.359392    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.376117 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.376090    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Sep 16 18:35:08.376250 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.376090    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Sep 16 18:35:08.376250 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.376125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-svgqf\""
Sep 16 18:35:08.376250 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.376097    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-dnj2m\""
Sep 16 18:35:08.376250 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.376090    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Sep 16 18:35:08.388647 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.388621    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:35:08.495111 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.495060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7j2d\" (UniqueName: \"kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.495298 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.495136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.596265 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.596231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7j2d\" (UniqueName: \"kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.596449 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.596306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.598896 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.598875    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.605814 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.605734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7j2d\" (UniqueName: \"kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.669340 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.669301    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:08.947324 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:08.947070    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:35:08.950122 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:35:08.950096    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62de581_7ac7_4fd1_9c31_59add7082717.slice/crio-4a70212dd8f04848b80af3e575e0480c94bfbf114495efe589bfb4d86061737d WatchSource:0}: Error finding container 4a70212dd8f04848b80af3e575e0480c94bfbf114495efe589bfb4d86061737d: Status 404 returned error can't find the container with id 4a70212dd8f04848b80af3e575e0480c94bfbf114495efe589bfb4d86061737d
Sep 16 18:35:09.598301 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:09.598259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" event={"ID":"e62de581-7ac7-4fd1-9c31-59add7082717","Type":"ContainerStarted","Data":"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"}
Sep 16 18:35:09.598815 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:09.598325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" event={"ID":"e62de581-7ac7-4fd1-9c31-59add7082717","Type":"ContainerStarted","Data":"4a70212dd8f04848b80af3e575e0480c94bfbf114495efe589bfb4d86061737d"}
Sep 16 18:35:09.598815 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:09.598395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:35:09.622986 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:09.622921    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podStartSLOduration=1.6228984899999999 podStartE2EDuration="1.62289849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:09.62264789 +0000 UTC m=+4232.852429576" watchObservedRunningTime="2026-09-16 18:35:09.62289849 +0000 UTC m=+4232.852680181"
Sep 16 18:35:40.604810 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:35:40.604777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:37:19.894182 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:19.894143    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:37:20.605256 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:20.605217    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:37:30.604674 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:30.604629    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:37:34.914119 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:34.914070    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" containerID="cri-o://dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38" gracePeriod=45
Sep 16 18:37:40.605449 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:40.605408    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:37:40.605918 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:40.605533    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:37:50.605030 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:37:50.604994    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:38:00.604657 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:00.604535    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:38:05.160314 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.160280    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:38:05.297203 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.297118    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7j2d\" (UniqueName: \"kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d\") pod \"e62de581-7ac7-4fd1-9c31-59add7082717\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") "
Sep 16 18:38:05.297368 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.297312    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs\") pod \"e62de581-7ac7-4fd1-9c31-59add7082717\" (UID: \"e62de581-7ac7-4fd1-9c31-59add7082717\") "
Sep 16 18:38:05.299343 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.299318    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e62de581-7ac7-4fd1-9c31-59add7082717" (UID: "e62de581-7ac7-4fd1-9c31-59add7082717"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:38:05.299447 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.299381    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d" (OuterVolumeSpecName: "kube-api-access-l7j2d") pod "e62de581-7ac7-4fd1-9c31-59add7082717" (UID: "e62de581-7ac7-4fd1-9c31-59add7082717"). InnerVolumeSpecName "kube-api-access-l7j2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:38:05.398459 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.398423    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7j2d\" (UniqueName: \"kubernetes.io/projected/e62de581-7ac7-4fd1-9c31-59add7082717-kube-api-access-l7j2d\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:38:05.398459 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.398456    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e62de581-7ac7-4fd1-9c31-59add7082717-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:38:05.424481 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.424447    2559 generic.go:356] "Generic (PLEG): container finished" podID="e62de581-7ac7-4fd1-9c31-59add7082717" containerID="dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38" exitCode=0
Sep 16 18:38:05.424634 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.424494    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"
Sep 16 18:38:05.424634 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.424540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" event={"ID":"e62de581-7ac7-4fd1-9c31-59add7082717","Type":"ContainerDied","Data":"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"}
Sep 16 18:38:05.424634 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.424627    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm" event={"ID":"e62de581-7ac7-4fd1-9c31-59add7082717","Type":"ContainerDied","Data":"4a70212dd8f04848b80af3e575e0480c94bfbf114495efe589bfb4d86061737d"}
Sep 16 18:38:05.424805 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.424660    2559 scope.go:117] "RemoveContainer" containerID="dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"
Sep 16 18:38:05.433572 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.433545    2559 scope.go:117] "RemoveContainer" containerID="dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"
Sep 16 18:38:05.433839 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:38:05.433821    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38\": container with ID starting with dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38 not found: ID does not exist" containerID="dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"
Sep 16 18:38:05.433886 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.433847    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38"} err="failed to get container status \"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38\": rpc error: code = NotFound desc = could not find container \"dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38\": container with ID starting with dd7ca55f0025b554000aec460bda6991b22093df0a62822968d0c267ba1bff38 not found: ID does not exist"
Sep 16 18:38:05.447601 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.447573    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:38:05.452932 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:05.452912    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c482d6gm"]
Sep 16 18:38:07.295597 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:07.295550    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" path="/var/lib/kubelet/pods/e62de581-7ac7-4fd1-9c31-59add7082717/volumes"
Sep 16 18:38:27.707896 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.707859    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:38:27.708353 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.708237    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main"
Sep 16 18:38:27.708353 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.708249    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main"
Sep 16 18:38:27.708353 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.708308    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e62de581-7ac7-4fd1-9c31-59add7082717" containerName="main"
Sep 16 18:38:27.710913 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.710896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.715475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.715447    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-26jvd\""
Sep 16 18:38:27.715681 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.715519    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Sep 16 18:38:27.715681 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.715542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-b9g2c\""
Sep 16 18:38:27.715806 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.715763    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Sep 16 18:38:27.716476 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.716462    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Sep 16 18:38:27.731854 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.731830    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:38:27.792762 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.792723    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2pg\" (UniqueName: \"kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.792952 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.792781    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.893526 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.893487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.893720 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.893592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2pg\" (UniqueName: \"kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.896067 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.896039    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:27.915466 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:27.915435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2pg\" (UniqueName: \"kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:28.021765 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.021682    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:28.299818 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.299790    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:38:28.301957 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:38:28.301927    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422a4159_e19a_4734_a27f_8db6a17e4fbe.slice/crio-60cb4a31af1eb260b2a19534ba67e83e29f648af23e068337eab1f21098ac0dc WatchSource:0}: Error finding container 60cb4a31af1eb260b2a19534ba67e83e29f648af23e068337eab1f21098ac0dc: Status 404 returned error can't find the container with id 60cb4a31af1eb260b2a19534ba67e83e29f648af23e068337eab1f21098ac0dc
Sep 16 18:38:28.537188 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.537151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" event={"ID":"422a4159-e19a-4734-a27f-8db6a17e4fbe","Type":"ContainerStarted","Data":"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"}
Sep 16 18:38:28.537372 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.537209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" event={"ID":"422a4159-e19a-4734-a27f-8db6a17e4fbe","Type":"ContainerStarted","Data":"60cb4a31af1eb260b2a19534ba67e83e29f648af23e068337eab1f21098ac0dc"}
Sep 16 18:38:28.537372 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.537287    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:38:28.556875 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:28.556761    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podStartSLOduration=1.556745888 podStartE2EDuration="1.556745888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:28.556009993 +0000 UTC m=+4431.785791685" watchObservedRunningTime="2026-09-16 18:38:28.556745888 +0000 UTC m=+4431.786527578"
Sep 16 18:38:59.544518 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:38:59.544487    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:39:37.618006 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:39:37.617978    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:39:37.635884 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:39:37.635854    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:40:43.774945 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:40:43.774910    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:40:49.544766 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:40:49.544726    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:40:58.794436 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:40:58.794399    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main" containerID="cri-o://ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6" gracePeriod=45
Sep 16 18:40:59.544682 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:40:59.544601    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:41:09.544762 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:09.544719    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:41:09.545250 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:09.544834    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:41:19.543865 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:19.543818    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:41:29.046864 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.046840    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:41:29.085465 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.085435    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs\") pod \"422a4159-e19a-4734-a27f-8db6a17e4fbe\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") "
Sep 16 18:41:29.085661 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.085502    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb2pg\" (UniqueName: \"kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg\") pod \"422a4159-e19a-4734-a27f-8db6a17e4fbe\" (UID: \"422a4159-e19a-4734-a27f-8db6a17e4fbe\") "
Sep 16 18:41:29.087836 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.087805    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "422a4159-e19a-4734-a27f-8db6a17e4fbe" (UID: "422a4159-e19a-4734-a27f-8db6a17e4fbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:41:29.088063 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.088043    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg" (OuterVolumeSpecName: "kube-api-access-lb2pg") pod "422a4159-e19a-4734-a27f-8db6a17e4fbe" (UID: "422a4159-e19a-4734-a27f-8db6a17e4fbe"). InnerVolumeSpecName "kube-api-access-lb2pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:29.186404 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.186369    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb2pg\" (UniqueName: \"kubernetes.io/projected/422a4159-e19a-4734-a27f-8db6a17e4fbe-kube-api-access-lb2pg\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:41:29.186404 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.186397    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/422a4159-e19a-4734-a27f-8db6a17e4fbe-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:41:29.371456 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.371370    2559 generic.go:356] "Generic (PLEG): container finished" podID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerID="ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6" exitCode=0
Sep 16 18:41:29.371456 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.371415    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" event={"ID":"422a4159-e19a-4734-a27f-8db6a17e4fbe","Type":"ContainerDied","Data":"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"}
Sep 16 18:41:29.371682 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.371458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr" event={"ID":"422a4159-e19a-4734-a27f-8db6a17e4fbe","Type":"ContainerDied","Data":"60cb4a31af1eb260b2a19534ba67e83e29f648af23e068337eab1f21098ac0dc"}
Sep 16 18:41:29.371682 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.371482    2559 scope.go:117] "RemoveContainer" containerID="ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"
Sep 16 18:41:29.371682 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.371419    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"
Sep 16 18:41:29.380159 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.380141    2559 scope.go:117] "RemoveContainer" containerID="ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"
Sep 16 18:41:29.380424 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:41:29.380404    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6\": container with ID starting with ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6 not found: ID does not exist" containerID="ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"
Sep 16 18:41:29.380492 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.380436    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6"} err="failed to get container status \"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6\": rpc error: code = NotFound desc = could not find container \"ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6\": container with ID starting with ebfa47b38412fc896ede4c90b8b2c0278e7d7d01735cf52c0e94c135419234e6 not found: ID does not exist"
Sep 16 18:41:29.398332 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.398304    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:41:29.402573 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:29.402534    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45bcgdr"]
Sep 16 18:41:31.295716 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:31.295685    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" path="/var/lib/kubelet/pods/422a4159-e19a-4734-a27f-8db6a17e4fbe/volumes"
Sep 16 18:41:41.729468 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.729432    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:41:41.730127 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.729823    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main"
Sep 16 18:41:41.730127 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.729836    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main"
Sep 16 18:41:41.730127 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.729917    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="422a4159-e19a-4734-a27f-8db6a17e4fbe" containerName="main"
Sep 16 18:41:41.734061 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.734041    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.737493 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.737468    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-epp-sa-dockercfg-hsvcj\""
Sep 16 18:41:41.737633 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.737493    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Sep 16 18:41:41.738629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.738609    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Sep 16 18:41:41.738629 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.738626    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Sep 16 18:41:41.738802 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.738766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-7wmck\""
Sep 16 18:41:41.744744 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.744721    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:41:41.784612 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.784548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdfh\" (UniqueName: \"kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.784775 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.784617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.885397 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.885361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdfh\" (UniqueName: \"kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.885646 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.885417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.888142 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.888109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:41.895895 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:41.895870    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdfh\" (UniqueName: \"kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:42.044896 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.044809    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:42.320072 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.320046    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:41:42.324838 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:41:42.324805    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3221fa_3d05_4585_9529_d94774b0eb44.slice/crio-948d8dbde55c20293b295f860a16b704eac8903e21057b22436850e814f6753e WatchSource:0}: Error finding container 948d8dbde55c20293b295f860a16b704eac8903e21057b22436850e814f6753e: Status 404 returned error can't find the container with id 948d8dbde55c20293b295f860a16b704eac8903e21057b22436850e814f6753e
Sep 16 18:41:42.431282 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.431245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" event={"ID":"fc3221fa-3d05-4585-9529-d94774b0eb44","Type":"ContainerStarted","Data":"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"}
Sep 16 18:41:42.431475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.431308    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" event={"ID":"fc3221fa-3d05-4585-9529-d94774b0eb44","Type":"ContainerStarted","Data":"948d8dbde55c20293b295f860a16b704eac8903e21057b22436850e814f6753e"}
Sep 16 18:41:42.431475 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.431350    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:41:42.454912 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:41:42.454856    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podStartSLOduration=1.454839271 podStartE2EDuration="1.454839271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:42.453458011 +0000 UTC m=+4625.683239728" watchObservedRunningTime="2026-09-16 18:41:42.454839271 +0000 UTC m=+4625.684620961"
Sep 16 18:42:13.437499 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:13.437467    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:42:19.801616 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.801583    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:19.804944 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.804925    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.817220 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.817200    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:19.923376 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.923554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923474    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8w4d\" (UniqueName: \"kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.923554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.923554 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.923695 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:19.923695 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:19.923639    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.024925 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.024873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.024925 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.024922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025156 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.024943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025156 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8w4d\" (UniqueName: \"kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025156 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025156 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025363 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025317    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025426 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025383    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.025527 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.025505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.027262 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.027235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.027425 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.027405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.035898 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.035870    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8w4d\" (UniqueName: \"kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-fmqvh\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.115819 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.115787    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:20.391258 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.391231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:20.393930 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:42:20.393903    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94458527_d47c_4ffa_ac2b_511dbe2da5c3.slice/crio-55b17116e742c6e28781d7148754f105a251f61f51fd3673ba020b404dcec588 WatchSource:0}: Error finding container 55b17116e742c6e28781d7148754f105a251f61f51fd3673ba020b404dcec588: Status 404 returned error can't find the container with id 55b17116e742c6e28781d7148754f105a251f61f51fd3673ba020b404dcec588
Sep 16 18:42:20.395892 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.395877    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:42:20.605278 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:20.605238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" event={"ID":"94458527-d47c-4ffa-ac2b-511dbe2da5c3","Type":"ContainerStarted","Data":"55b17116e742c6e28781d7148754f105a251f61f51fd3673ba020b404dcec588"}
Sep 16 18:42:22.616834 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:22.616790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" event={"ID":"94458527-d47c-4ffa-ac2b-511dbe2da5c3","Type":"ContainerStarted","Data":"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"}
Sep 16 18:42:22.638326 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:22.638275    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" podStartSLOduration=2.153719789 podStartE2EDuration="3.638257825s" totalImagesPullingTime="1.484538036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:22.6373059 +0000 UTC m=+4665.867087591" watchObservedRunningTime="2026-09-16 18:42:22.638257825 +0000 UTC m=+4665.868039515"
Sep 16 18:42:23.116079 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:23.116044    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:30.116125 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:30.116035    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:30.128642 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:30.128619    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:30.670915 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:30.670884    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:39.537837 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.537800    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:42:39.540210 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.540179    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:39.548796 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:42:39.548758    2559 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 16 18:42:39.548796 ip-10-0-142-164 kubenswrapper[2559]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 18:42:39.548796 ip-10-0-142-164 kubenswrapper[2559]:         , stdout: , stderr: , exit code -1
Sep 16 18:42:39.548796 ip-10-0-142-164 kubenswrapper[2559]:  > containerID="72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813" cmd=["/bin/sleep","15"]
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.548817    2559 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" message="" err=<
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:         , stdout: , stderr: , exit code -1
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:  >
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:42:39.548830    2559 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:         , stdout: , stderr: , exit code -1
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main" containerID="cri-o://72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"
Sep 16 18:42:39.549019 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.548877    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main" containerID="cri-o://72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813" gracePeriod=60
Sep 16 18:42:39.660494 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.660454    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": dial tcp 10.132.0.53:8000: connect: connection refused"
Sep 16 18:42:39.894728 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.894704    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:39.989551 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989518    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989739 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989579    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989739 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989600    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8w4d\" (UniqueName: \"kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989739 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989622    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989739 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989959 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989762    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs\") pod \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\" (UID: \"94458527-d47c-4ffa-ac2b-511dbe2da5c3\") "
Sep 16 18:42:39.989959 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989808    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:42:39.989959 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989821    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:42:39.989959 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.989936    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home" (OuterVolumeSpecName: "home") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:42:39.990141 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.990024    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tmp-dir\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:39.990141 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.990039    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-model-cache\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:39.990141 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.990047    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-home\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:39.991896 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.991855    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:42:39.991995 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.991923    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm" (OuterVolumeSpecName: "dshm") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:42:39.991995 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:39.991941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d" (OuterVolumeSpecName: "kube-api-access-s8w4d") pod "94458527-d47c-4ffa-ac2b-511dbe2da5c3" (UID: "94458527-d47c-4ffa-ac2b-511dbe2da5c3"). InnerVolumeSpecName "kube-api-access-s8w4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:42:40.091119 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.091034    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8w4d\" (UniqueName: \"kubernetes.io/projected/94458527-d47c-4ffa-ac2b-511dbe2da5c3-kube-api-access-s8w4d\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:40.091119 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.091061    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94458527-d47c-4ffa-ac2b-511dbe2da5c3-dshm\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:40.091119 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.091071    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94458527-d47c-4ffa-ac2b-511dbe2da5c3-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:42:40.709019 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.708981    2559 generic.go:356] "Generic (PLEG): container finished" podID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerID="72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813" exitCode=0
Sep 16 18:42:40.709410 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.709046    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"
Sep 16 18:42:40.709410 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.709064    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" event={"ID":"94458527-d47c-4ffa-ac2b-511dbe2da5c3","Type":"ContainerDied","Data":"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"}
Sep 16 18:42:40.709410 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.709153    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh" event={"ID":"94458527-d47c-4ffa-ac2b-511dbe2da5c3","Type":"ContainerDied","Data":"55b17116e742c6e28781d7148754f105a251f61f51fd3673ba020b404dcec588"}
Sep 16 18:42:40.709410 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.709184    2559 scope.go:117] "RemoveContainer" containerID="72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"
Sep 16 18:42:40.718413 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.718395    2559 scope.go:117] "RemoveContainer" containerID="72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"
Sep 16 18:42:40.718730 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:42:40.718709    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813\": container with ID starting with 72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813 not found: ID does not exist" containerID="72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"
Sep 16 18:42:40.718790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.718739    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813"} err="failed to get container status \"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813\": rpc error: code = NotFound desc = could not find container \"72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813\": container with ID starting with 72ae022689c4f55b6e842787f5cb66f6568586fa6dc7b7a3fc0ac5ebe1a8a813 not found: ID does not exist"
Sep 16 18:42:40.733014 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.732989    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:40.736988 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:40.736965    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-fmqvh"]
Sep 16 18:42:41.295780 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:41.295745    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" path="/var/lib/kubelet/pods/94458527-d47c-4ffa-ac2b-511dbe2da5c3/volumes"
Sep 16 18:42:43.437395 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:43.437350    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:42:53.438091 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:53.438053    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:42:54.557778 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:42:54.557729    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" containerID="cri-o://11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad" gracePeriod=45
Sep 16 18:43:03.437383 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:03.437339    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:43:03.437780 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:03.437450    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:43:13.437612 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:13.437550    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:43:23.437120 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:23.437076    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 18:43:24.811294 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.811267    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:43:24.890368 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.890272    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs\") pod \"fc3221fa-3d05-4585-9529-d94774b0eb44\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") "
Sep 16 18:43:24.890544 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.890371    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sdfh\" (UniqueName: \"kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh\") pod \"fc3221fa-3d05-4585-9529-d94774b0eb44\" (UID: \"fc3221fa-3d05-4585-9529-d94774b0eb44\") "
Sep 16 18:43:24.892455 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.892429    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc3221fa-3d05-4585-9529-d94774b0eb44" (UID: "fc3221fa-3d05-4585-9529-d94774b0eb44"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:24.892543 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.892448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh" (OuterVolumeSpecName: "kube-api-access-2sdfh") pod "fc3221fa-3d05-4585-9529-d94774b0eb44" (UID: "fc3221fa-3d05-4585-9529-d94774b0eb44"). InnerVolumeSpecName "kube-api-access-2sdfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:24.922957 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.922927    2559 generic.go:356] "Generic (PLEG): container finished" podID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerID="11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad" exitCode=0
Sep 16 18:43:24.923067 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.922972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" event={"ID":"fc3221fa-3d05-4585-9529-d94774b0eb44","Type":"ContainerDied","Data":"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"}
Sep 16 18:43:24.923067 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.922976    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"
Sep 16 18:43:24.923067 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.923015    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr" event={"ID":"fc3221fa-3d05-4585-9529-d94774b0eb44","Type":"ContainerDied","Data":"948d8dbde55c20293b295f860a16b704eac8903e21057b22436850e814f6753e"}
Sep 16 18:43:24.923067 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.923036    2559 scope.go:117] "RemoveContainer" containerID="11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"
Sep 16 18:43:24.931860 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.931843    2559 scope.go:117] "RemoveContainer" containerID="11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"
Sep 16 18:43:24.932135 ip-10-0-142-164 kubenswrapper[2559]: E0916 18:43:24.932120    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad\": container with ID starting with 11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad not found: ID does not exist" containerID="11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"
Sep 16 18:43:24.932185 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.932141    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad"} err="failed to get container status \"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad\": rpc error: code = NotFound desc = could not find container \"11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad\": container with ID starting with 11a1706bbbe92985941e5e2d29261e3e45cf3c1502a82906958ec97a14b112ad not found: ID does not exist"
Sep 16 18:43:24.946534 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.946509    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:43:24.950303 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.950280    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-6f5gr"]
Sep 16 18:43:24.990960 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.990932    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sdfh\" (UniqueName: \"kubernetes.io/projected/fc3221fa-3d05-4585-9529-d94774b0eb44-kube-api-access-2sdfh\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:43:24.990960 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:24.990957    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3221fa-3d05-4585-9529-d94774b0eb44-tls-certs\") on node \"ip-10-0-142-164.ec2.internal\" DevicePath \"\""
Sep 16 18:43:25.295730 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:25.295651    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" path="/var/lib/kubelet/pods/fc3221fa-3d05-4585-9529-d94774b0eb44/volumes"
Sep 16 18:43:53.501978 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:53.501947    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k4rtb_2b435325-ea04-4231-8d85-a3c6cdfd52fe/discovery/0.log"
Sep 16 18:43:54.394602 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:54.394542    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k4rtb_2b435325-ea04-4231-8d85-a3c6cdfd52fe/discovery/0.log"
Sep 16 18:43:56.020654 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:43:56.020619    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-8vm4x_238a3d8c-dea6-4b4e-b89e-3e27447d9901/manager/0.log"
Sep 16 18:44:05.162218 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:05.162126    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7c82q_8cea32ac-e0c2-4c23-8873-acdedd71424b/global-pull-secret-syncer/0.log"
Sep 16 18:44:05.274308 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:05.274261    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cfvdg_05bcc847-b121-40de-86f0-bc6dbafa1731/konnectivity-agent/0.log"
Sep 16 18:44:05.390831 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:05.390802    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-164.ec2.internal_174cc1912109f9f57a175c779d7e35e4/haproxy/0.log"
Sep 16 18:44:10.464005 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:10.463970    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-8vm4x_238a3d8c-dea6-4b4e-b89e-3e27447d9901/manager/0.log"
Sep 16 18:44:11.863206 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.863172    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/alertmanager/0.log"
Sep 16 18:44:11.885617 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.885589    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/config-reloader/0.log"
Sep 16 18:44:11.907293 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.907260    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/kube-rbac-proxy-web/0.log"
Sep 16 18:44:11.929221 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.929193    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/kube-rbac-proxy/0.log"
Sep 16 18:44:11.951485 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.951458    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/kube-rbac-proxy-metric/0.log"
Sep 16 18:44:11.976856 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.976836    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/prom-label-proxy/0.log"
Sep 16 18:44:11.999100 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:11.999069    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ce2394d0-4784-4a22-8477-8421ec5b98fb/init-config-reloader/0.log"
Sep 16 18:44:12.068615 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.068590    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-mhwg8_ca70a60f-0ced-4278-a8f9-02518e3dbe26/cluster-monitoring-operator/0.log"
Sep 16 18:44:12.099788 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.099758    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-dq45v_72c5e7b4-fa73-485a-992c-232f470fc737/kube-state-metrics/0.log"
Sep 16 18:44:12.122721 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.122654    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-dq45v_72c5e7b4-fa73-485a-992c-232f470fc737/kube-rbac-proxy-main/0.log"
Sep 16 18:44:12.148520 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.148494    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-dq45v_72c5e7b4-fa73-485a-992c-232f470fc737/kube-rbac-proxy-self/0.log"
Sep 16 18:44:12.183494 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.183471    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56d6969896-gts96_020d37a4-e265-4359-b111-a5373aa2f21e/metrics-server/0.log"
Sep 16 18:44:12.391883 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.391794    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kb2wt_44f422e4-8f77-4742-8f5b-0ad0dff980e6/node-exporter/0.log"
Sep 16 18:44:12.421843 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.421814    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kb2wt_44f422e4-8f77-4742-8f5b-0ad0dff980e6/kube-rbac-proxy/0.log"
Sep 16 18:44:12.458229 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.458205    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kb2wt_44f422e4-8f77-4742-8f5b-0ad0dff980e6/init-textfile/0.log"
Sep 16 18:44:12.730633 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.730602    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/prometheus/0.log"
Sep 16 18:44:12.761599 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.761551    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/config-reloader/0.log"
Sep 16 18:44:12.786992 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.786965    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/thanos-sidecar/0.log"
Sep 16 18:44:12.809821 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.809793    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/kube-rbac-proxy-web/0.log"
Sep 16 18:44:12.843058 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.843032    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/kube-rbac-proxy/0.log"
Sep 16 18:44:12.868062 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.868036    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/kube-rbac-proxy-thanos/0.log"
Sep 16 18:44:12.894842 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.894822    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_01b287a8-ec65-4317-bccb-715e8c03f3a0/init-config-reloader/0.log"
Sep 16 18:44:12.930058 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.929967    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-74mzz_98b86887-846b-4c5f-a7be-91ec234c1385/prometheus-operator/0.log"
Sep 16 18:44:12.952428 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:12.952407    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-74mzz_98b86887-846b-4c5f-a7be-91ec234c1385/kube-rbac-proxy/0.log"
Sep 16 18:44:14.419425 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:14.419385    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-jl5w5_1b5a31bf-d965-41a6-8e2d-57a5582241a9/networking-console-plugin/0.log"
Sep 16 18:44:14.915402 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:14.915375    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/1.log"
Sep 16 18:44:14.923508 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:14.923485    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wg9q8_e742e177-ac18-44c6-8521-7e549c2c1a54/console-operator/2.log"
Sep 16 18:44:15.171059 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.170982    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"]
Sep 16 18:44:15.171369 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171355    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main"
Sep 16 18:44:15.171465 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171371    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main"
Sep 16 18:44:15.171465 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171393    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main"
Sep 16 18:44:15.171465 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171398    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main"
Sep 16 18:44:15.171639 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171471    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="94458527-d47c-4ffa-ac2b-511dbe2da5c3" containerName="main"
Sep 16 18:44:15.171639 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.171481    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3221fa-3d05-4585-9529-d94774b0eb44" containerName="main"
Sep 16 18:44:15.174534 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.174508    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.177732 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.177714    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wzhjv\"/\"kube-root-ca.crt\""
Sep 16 18:44:15.179134 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.179114    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wzhjv\"/\"default-dockercfg-d5znf\""
Sep 16 18:44:15.179233 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.179134    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wzhjv\"/\"openshift-service-ca.crt\""
Sep 16 18:44:15.185473 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.185450    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"]
Sep 16 18:44:15.236221 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.236189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-podres\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.236398 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.236230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-sys\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.236398 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.236259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwj2n\" (UniqueName: \"kubernetes.io/projected/8902116c-6e5c-4e14-b85f-864c83ba1296-kube-api-access-wwj2n\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.236398 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.236356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-lib-modules\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.236398 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.236393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-proc\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337498 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-podres\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337498 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-sys\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337529    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwj2n\" (UniqueName: \"kubernetes.io/projected/8902116c-6e5c-4e14-b85f-864c83ba1296-kube-api-access-wwj2n\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-lib-modules\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-sys\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-podres\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-proc\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-proc\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.337790 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.337763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8902116c-6e5c-4e14-b85f-864c83ba1296-lib-modules\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.347085 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.347061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwj2n\" (UniqueName: \"kubernetes.io/projected/8902116c-6e5c-4e14-b85f-864c83ba1296-kube-api-access-wwj2n\") pod \"perf-node-gather-daemonset-94b2t\" (UID: \"8902116c-6e5c-4e14-b85f-864c83ba1296\") " pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.484225 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.484129    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:15.757498 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:15.757394    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"]
Sep 16 18:44:15.760408 ip-10-0-142-164 kubenswrapper[2559]: W0916 18:44:15.760380    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8902116c_6e5c_4e14_b85f_864c83ba1296.slice/crio-a049e51a273e6fb6dcda10855c13f6867ef64d02801d97443709a67ca3b12881 WatchSource:0}: Error finding container a049e51a273e6fb6dcda10855c13f6867ef64d02801d97443709a67ca3b12881: Status 404 returned error can't find the container with id a049e51a273e6fb6dcda10855c13f6867ef64d02801d97443709a67ca3b12881
Sep 16 18:44:16.049589 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:16.049487    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-hgfzz_c3bb428a-1f57-4e6f-b606-4d60fef7f8b6/volume-data-source-validator/0.log"
Sep 16 18:44:16.157753 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:16.157712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t" event={"ID":"8902116c-6e5c-4e14-b85f-864c83ba1296","Type":"ContainerStarted","Data":"26f6bb352b2ec51985cb6eca3dfec756689c82ed55a2d2f861deefa27d0d7af5"}
Sep 16 18:44:16.157917 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:16.157786    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:16.157917 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:16.157812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t" event={"ID":"8902116c-6e5c-4e14-b85f-864c83ba1296","Type":"ContainerStarted","Data":"a049e51a273e6fb6dcda10855c13f6867ef64d02801d97443709a67ca3b12881"}
Sep 16 18:44:16.177399 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:16.177351    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t" podStartSLOduration=1.177337282 podStartE2EDuration="1.177337282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:16.176765217 +0000 UTC m=+4779.406546920" watchObservedRunningTime="2026-09-16 18:44:16.177337282 +0000 UTC m=+4779.407118973"
Sep 16 18:44:17.305351 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:17.305316    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zsd8k_4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9/dns/0.log"
Sep 16 18:44:17.347211 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:17.347176    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zsd8k_4f5b0bd3-e360-41f1-a3a1-c3da0ef9ffe9/kube-rbac-proxy/0.log"
Sep 16 18:44:17.460576 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:17.460528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndng9_1909d2d2-d9ad-4568-a352-55f882e0f6cb/dns-node-resolver/0.log"
Sep 16 18:44:18.131545 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:18.131505    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hqchc_b59791de-9fae-4f5f-8a78-5cb47514ee7e/node-ca/0.log"
Sep 16 18:44:19.314447 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:19.314417    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k4rtb_2b435325-ea04-4231-8d85-a3c6cdfd52fe/discovery/0.log"
Sep 16 18:44:19.895969 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:19.895934    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j79n7_e888ce83-3f2c-4c6a-ab52-23291c51fd24/serve-healthcheck-canary/0.log"
Sep 16 18:44:20.553522 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:20.553493    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsksl_acd362b9-234a-43cb-ae75-8bdc130041f2/kube-rbac-proxy/0.log"
Sep 16 18:44:20.575289 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:20.575261    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsksl_acd362b9-234a-43cb-ae75-8bdc130041f2/exporter/0.log"
Sep 16 18:44:20.597762 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:20.597732    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsksl_acd362b9-234a-43cb-ae75-8bdc130041f2/extractor/0.log"
Sep 16 18:44:22.175002 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:22.174974    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wzhjv/perf-node-gather-daemonset-94b2t"
Sep 16 18:44:23.782769 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:23.782723    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86b645b5b5-bgjcv_97bcbee0-ad5a-4cd8-b894-bdeb1fe3c687/manager/0.log"
Sep 16 18:44:24.735302 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:24.735261    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k4rtb_2b435325-ea04-4231-8d85-a3c6cdfd52fe/discovery/0.log"
Sep 16 18:44:26.238902 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:26.238869    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k4rtb_2b435325-ea04-4231-8d85-a3c6cdfd52fe/discovery/0.log"
Sep 16 18:44:31.547942 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:31.547902    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-4pnwk_9018820e-0403-4bcc-bb44-db4942cc15af/migrator/0.log"
Sep 16 18:44:31.570831 ip-10-0-142-164 kubenswrapper[2559]: I0916 18:44:31.570797    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-4pnwk_9018820e-0403-4bcc-bb44-db4942cc15af/graceful-termination/0.log"