Sep 18 20:18:25.010655 ip-10-0-140-180 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 20:18:25.010669 ip-10-0-140-180 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 20:18:25.010679 ip-10-0-140-180 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 20:18:25.010997 ip-10-0-140-180 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 20:18:35.213669 ip-10-0-140-180 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 18 20:18:35.213685 ip-10-0-140-180 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ce9eceb77d414778898a63e8fc9d69ae --
Sep 18 20:20:26.105012 ip-10-0-140-180 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 20:20:29.609974 ip-10-0-140-180 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:29.609974 ip-10-0-140-180 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 20:20:29.609974 ip-10-0-140-180 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:29.609974 ip-10-0-140-180 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 20:20:29.609974 ip-10-0-140-180 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:29.610801 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.610445    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 20:20:29.614066 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614050    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.614066 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614066    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614070    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614074    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614078    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614081    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614084    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614087    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614092    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614095    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614098    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614101    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614105    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614108    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614111    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614114    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614119    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614122    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614126    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614129    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614132    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.614131 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614135    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614138    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614141    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614144    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614147    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614150    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614153    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614156    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614158    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614161    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614164    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614166    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614169    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614172    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614177    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614181    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614185    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614188    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614192    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.614653 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614196    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614199    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614202    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614205    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614208    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614211    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614214    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614217    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614220    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614223    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614226    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614229    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614232    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614234    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614237    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614241    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614245    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614248    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614250    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.615494 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614253    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614256    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614262    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614265    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614269    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614272    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614275    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614279    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614282    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614286    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614289    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614292    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614295    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614299    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614302    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614305    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614307    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614310    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614313    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614317    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.616314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614319    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614322    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614325    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614329    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614332    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614334    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614338    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614340    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614343    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614346    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614349    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614352    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614355    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614358    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614360    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614363    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614366    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614368    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614371    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.614373    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.617250 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616163    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616181    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616187    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616197    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616205    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616210    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616215    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616221    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616226    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616231    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616236    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616241    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616248    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616255    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616260    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616265    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616269    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616275    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616279    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.618227 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616284    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616294    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616299    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616303    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616308    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616312    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616316    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616320    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616325    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616329    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616335    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616343    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616348    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616353    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616359    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616367    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616372    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616377    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616381    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.619163 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616385    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616390    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616394    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616399    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616403    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616408    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616412    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616417    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616422    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616426    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616431    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616435    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616441    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616446    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616451    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616455    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616460    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616464    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616469    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.619977 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616473    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616478    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616482    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616487    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616491    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616496    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616500    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616505    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616513    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616518    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616523    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616527    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616532    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616537    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616541    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616546    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616551    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616555    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616559    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616564    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.620598 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616569    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616573    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616577    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616582    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616589    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616594    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616599    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616604    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616608    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616613    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616617    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616624    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616629    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616633    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616638    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616642    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616646    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616651    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616655    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616660    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.621197 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616666    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.616671    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617477    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617491    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617498    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617504    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617510    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617515    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617520    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617525    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617530    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617536    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617541    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617546    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617550    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617554    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617559    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617564    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617569    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617574    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.621889 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617578    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617583    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617587    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617594    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617599    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617603    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617608    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617613    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617617    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617621    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617626    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617630    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617636    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617640    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617645    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617649    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617654    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617658    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617663    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617667    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.622565 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617672    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617676    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617680    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617685    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617689    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617694    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617698    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617703    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617721    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617726    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617730    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617736    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617740    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617745    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617750    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617756    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617762    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617767    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617771    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617776    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.623123 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617780    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617785    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617790    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617794    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617801    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617808    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617813    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617817    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617822    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617827    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617838    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617843    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617847    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617852    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617857    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617862    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617866    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617870    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617875    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.623766 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617879    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617884    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617888    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617893    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617899    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617904    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617909    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617913    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617920    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617925    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617930    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617934    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617939    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617944    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617950    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617957    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617963    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617967    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617973    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617978    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.624282 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617984    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.617988    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618101    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618152    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618162    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618169    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618177    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618182    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618190    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618204    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618210    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618215    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618221    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618226    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618231    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618236    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618240    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618246    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618252    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618257    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618267    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618273    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618278    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618284    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 20:20:29.624836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618290    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618296    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618301    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618306    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618312    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618317    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618322    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618327    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618333    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618338    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618344    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618349    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618354    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618359    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618365    2564 flags.go:64] FLAG: --enable-server="true"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618370    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618377    2564 flags.go:64] FLAG: --event-burst="100"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618382    2564 flags.go:64] FLAG: --event-qps="50"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618387    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618392    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618397    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618404    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618409    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618414    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618419    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 18 20:20:29.625475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618425    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618430    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618436    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618444    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618451    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618457    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618462    2564 flags.go:64] FLAG: --feature-gates=""
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618468    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618473    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618478    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618483    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618489    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618494    2564 flags.go:64] FLAG: --help="false"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618500    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618506    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618511    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618516    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618522    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618528    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618533    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618538    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618543    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618548    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 20:20:29.626147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618553    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618558    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618563    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618568    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618573    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618578    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618583    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618588    2564 flags.go:64] FLAG: --lock-file=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618593    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618598    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618603    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618619    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618627    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618634    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618639    2564 flags.go:64] FLAG: --logging-format="text"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618644    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618650    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618655    2564 flags.go:64] FLAG: --manifest-url=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618660    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618667    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618673    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618680    2564 flags.go:64] FLAG: --max-pods="110"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618685    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618690    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618695    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 20:20:29.626917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618700    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618728    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618733    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618737    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618750    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618755    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618760    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618765    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618770    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618779    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618784    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618789    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618795    2564 flags.go:64] FLAG: --port="10250"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618801    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618805    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059f4e69855b91b00"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618811    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618816    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618821    2564 flags.go:64] FLAG: --register-node="true"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618826    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618833    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618842    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618848    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618853    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618860    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618865    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 20:20:29.627593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618870    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618876    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618881    2564 flags.go:64] FLAG: --runonce="false"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618886    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618891    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618896    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618901    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618906    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618911    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618916    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618922    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618926    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618931    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618936    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618941    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618947    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618952    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618957    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618965    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618970    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618975    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618983    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618987    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618993    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.618998    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 20:20:29.628281 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619003    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619008    2564 flags.go:64] FLAG: --v="2"
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619018    2564 flags.go:64] FLAG: --version="false"
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619025    2564 flags.go:64] FLAG: --vmodule=""
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619031    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.619037    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619263    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619272    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619280    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619286    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619292    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619297    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619302    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619307    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619312    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619317    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619322    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619327    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619332    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619337    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.628964 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619341    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619346    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619351    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619355    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619359    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619364    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619369    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619373    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619377    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619381    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619386    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619390    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619395    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619400    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619407    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619412    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619417    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619422    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619427    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619431    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.629508 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619436    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619440    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619448    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619454    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619459    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619464    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619469    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619473    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619479    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619484    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619489    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619494    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619499    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619504    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619509    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619513    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619517    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619522    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619526    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619531    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.630048 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619535    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619540    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619544    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619549    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619553    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619558    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619566    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619571    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619576    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619581    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619586    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619591    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619595    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619600    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619604    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619608    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619612    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619617    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619622    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619626    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.630618 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619631    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619635    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619640    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619644    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619649    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619653    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619658    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619662    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619667    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619672    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619676    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619681    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619685    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619690    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619694    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619699    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619723    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619729    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619737    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.631264 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619742    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619747    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619752    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619758    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619763    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.619768    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.620320    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628027    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628044    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628065    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628071    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628127    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628133    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628136    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628140    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.631796 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628143    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628146    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628149    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628152    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628155    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628158    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628161    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628166    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628170    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628174    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628177    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628180    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628183    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628186    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628190    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628193    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628197    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628200    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628203    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.632208 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628206    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628209    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628212    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628215    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628218    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628222    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628225    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628228    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628232    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628234    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628237    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628240    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628243    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628246    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628249    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628252    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628255    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628258    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628260    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628264    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.632700 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628266    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628269    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628272    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628275    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628278    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628281    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628284    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628287    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628292    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628296    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628299    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628302    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628306    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628309    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628313    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628316    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628319    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628322    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628325    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.633254 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628328    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628331    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628334    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628337    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628340    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628343    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628346    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628349    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628351    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628354    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628357    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628360    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628363    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628366    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628368    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628371    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628374    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628377    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628380    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628383    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.633778 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628386    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628389    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628392    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628395    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628398    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628400    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628405    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628408    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628411    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628415    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628418    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628421    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628424    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628427    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628429    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628432    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.634302 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628435    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628441    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628558    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628562    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628567    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628571    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628575    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628577    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628580    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628584    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628587    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628590    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628593    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628596    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:29.634763 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628599    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628603    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628607    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628611    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628614    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628617    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628620    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628623    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628626    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628629    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628632    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628635    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628638    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628641    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628643    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628646    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628649    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628652    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628655    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:29.635130 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628657    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628660    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628663    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628666    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628668    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628671    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628674    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628676    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628679    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628682    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628685    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628687    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628690    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628693    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628696    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628698    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628701    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628724    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628727    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628730    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:29.635646 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628733    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628736    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628739    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628741    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628744    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628752    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628755    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628758    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628761    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628764    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628767    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628769    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628772    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628775    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628778    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628781    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628784    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628786    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628789    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628792    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:29.636288 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628795    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628798    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628801    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628804    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628807    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628809    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628812    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628815    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628818    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628821    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628824    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628827    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628830    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628833    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628835    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628838    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628841    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628844    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628847    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628850    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:29.636867 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628853    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628856    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628858    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628861    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628864    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628866    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628869    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:29.628873    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628879    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.628983    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.631698    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.632541    2564 server.go:1019] "Starting client certificate rotation"
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.632656    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:20:29.637610 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.632695    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:20:29.637982 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.634856    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:20:29.637982 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.636696    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:20:29.645414 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.645388    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 18 20:20:29.648855 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.648838    2564 log.go:25] "Validated CRI v1 image API"
Sep 18 20:20:29.649759 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.649743    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 20:20:29.651968 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.651946    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 99d7b539-235a-4ffb-a97f-a5cdc56766a2:/dev/nvme0n1p3 de688bb1-0dc9-4923-a29a-de61ca0ae301:/dev/nvme0n1p4]
Sep 18 20:20:29.652053 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.651966    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 20:20:29.657135 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.657013    2564 manager.go:216] Machine: {Timestamp:2026-09-18 20:20:29.656471963 +0000 UTC m=+2.005744372 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100250 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b50886a509766988b1324617bee92 SystemUUID:ec2b5088-6a50-9766-988b-1324617bee92 BootID:ce9eceb7-7d41-4778-898a-63e8fc9d69ae Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f4:8d:9d:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f4:8d:9d:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:fe:b3:ad:cf:5e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 20:20:29.657135 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.657130    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 20:20:29.657264 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.657208    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 20:20:29.658800 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.658775    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 20:20:29.658949 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.658806    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 20:20:29.659957 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.659946    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 20:20:29.660000 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.659960    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 20:20:29.660000 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.659968    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:20:29.660000 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.659980    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:20:29.661291 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.661280    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 20:20:29.664346 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.664329    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:20:29.665213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665200    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:20:29.665337 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665327    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 20:20:29.665382 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665359    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 20:20:29.665382 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665369    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 20:20:29.665462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665384    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 20:20:29.665462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665394    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 18 20:20:29.665462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665403    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 20:20:29.665828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665808    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:20:29.665878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.665834    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:20:29.672033 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.672006    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 20:20:29.674679 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.674652    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 20:20:29.674764 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.674691    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 20:20:29.675947 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.675933    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 20:20:29.675991 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.675959    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 20:20:29.675991 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.675969    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 20:20:29.675991 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.675980    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676000    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676011    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676040    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676056    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676068    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 20:20:29.676080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676077    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 20:20:29.676254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676089    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 20:20:29.676254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676095    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 20:20:29.676254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676131    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 20:20:29.676254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.676137    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 20:20:29.679624 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.679606    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 20:20:29.679692 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.679640    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 20:20:29.679692 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.679617    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 20:20:29.680524 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.680509    2564 server.go:1268] "Started kubelet"
Sep 18 20:20:29.680600 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.680569    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 20:20:29.680767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.680656    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 20:20:29.680813 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.680795    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 20:20:29.681646 ip-10-0-140-180 systemd[1]: Started Kubernetes Kubelet.
Sep 18 20:20:29.682124 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.682101    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 20:20:29.714097 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.714069    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4fcq2"
Sep 18 20:20:29.714337 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.714318    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4fcq2"
Sep 18 20:20:29.714912 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.714891    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 20:20:29.715199 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.715169    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 20:20:29.718625 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.717829    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-180.ec2.internal.18d683cf9f4c7832  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-180.ec2.internal,UID:ip-10-0-140-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-180.ec2.internal,},FirstTimestamp:2026-09-18 20:20:29.680408626 +0000 UTC m=+2.029681013,LastTimestamp:2026-09-18 20:20:29.680408626 +0000 UTC m=+2.029681013,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-180.ec2.internal,}"
Sep 18 20:20:29.721915 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.721898    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 20:20:29.722013 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.721920    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 20:20:29.722143 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.722030    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:29.722557 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722534    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 20:20:29.722665 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722592    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 20:20:29.722665 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722609    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 20:20:29.722828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722811    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 20:20:29.722828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722825    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 20:20:29.723001 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722953    2564 factory.go:55] Registering systemd factory
Sep 18 20:20:29.723001 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.722970    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 18 20:20:29.723240 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723221    2564 factory.go:153] Registering CRI-O factory
Sep 18 20:20:29.723240 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723237    2564 factory.go:223] Registration of the crio container factory successfully
Sep 18 20:20:29.723310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723281    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 20:20:29.723310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723295    2564 factory.go:103] Registering Raw factory
Sep 18 20:20:29.723310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723305    2564 manager.go:1196] Started watching for new ooms in manager
Sep 18 20:20:29.723943 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.723931    2564 manager.go:349] Starting recovery of all containers
Sep 18 20:20:29.732415 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.732280    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:29.735769 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.735739    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-180.ec2.internal\" not found" node="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.735953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.735939    2564 manager.go:354] Recovery completed
Sep 18 20:20:29.739878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.739866    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.741528 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.741513    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.741607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.741541    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.741607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.741557    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.742114 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742099    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 20:20:29.742114 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742112    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 20:20:29.742239 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742131    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:20:29.742239 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742151    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.742955 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742939    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.743010 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742968    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.743010 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.742981    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.744397 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.744381    2564 policy_none.go:49] "None policy: Start"
Sep 18 20:20:29.744446 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.744407    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 20:20:29.744446 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.744422    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 20:20:29.746698 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.746684    2564 policy_none.go:47] "Start"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783320    2564 manager.go:337] "Starting Device Plugin manager"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.783354    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783364    2564 server.go:87] "Starting device plugin registration server"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783587    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783610    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783708    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783789    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.783798    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.784499    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.784552    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.786026    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.787333    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.787378    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.787398    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.787438    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 20:20:29.792116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.789795    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:29.884484 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.884412    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.887238 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887220    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.887323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887254    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.887323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887268    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.887323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887300    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.887569 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887539    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"]
Sep 18 20:20:29.887626 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.887619    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.888540 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.888523    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.888608 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.888551    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.888608 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.888562    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.891719 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.891690    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.891861 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.891844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.891926 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.891884    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.892613 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892597    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.892695 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892621    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.892695 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892631    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.892695 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892670    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.892695 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892691    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.892862 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.892701    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.894159 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.894142    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.894253 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.894162    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-180.ec2.internal\": node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:29.895040 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.895023    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.895132 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.895054    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:29.896819 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.896801    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:29.896915 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.896829    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:29.896915 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.896843    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:29.916622 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.916599    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:29.923986 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.923969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/076cef5425bde9780a5fcdde6e01c168-config\") pod \"kube-apiserver-proxy-ip-10-0-140-180.ec2.internal\" (UID: \"076cef5425bde9780a5fcdde6e01c168\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.924062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.923992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.924062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:29.924010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.928357 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.928340    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-180.ec2.internal\" not found" node="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:29.933312 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:29.933296    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-180.ec2.internal\" not found" node="ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.017666 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.017646    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.025133 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.025230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025148    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.025230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/076cef5425bde9780a5fcdde6e01c168-config\") pod \"kube-apiserver-proxy-ip-10-0-140-180.ec2.internal\" (UID: \"076cef5425bde9780a5fcdde6e01c168\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.025326 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.025326 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/076cef5425bde9780a5fcdde6e01c168-config\") pod \"kube-apiserver-proxy-ip-10-0-140-180.ec2.internal\" (UID: \"076cef5425bde9780a5fcdde6e01c168\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.025326 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.025229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a27b89508e8bec15507dad5bcf40fdba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal\" (UID: \"a27b89508e8bec15507dad5bcf40fdba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.118921 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.118893    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.220909 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.220832    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.232056 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.232036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.238288 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.237832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:30.326369 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.326342    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.429511 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.429485    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.530441 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.530366    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.564240 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.564213    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:30.631349 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.631320    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.632502 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.632483    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 20:20:30.632671 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.632610    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:20:30.632737 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.632662    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:20:30.632737 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.632662    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:20:30.721707 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.721665    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 20:15:29 +0000 UTC" deadline="2028-04-19 08:03:46.842009616 +0000 UTC"
Sep 18 20:20:30.721707 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.721700    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13883h43m16.120313083s"
Sep 18 20:20:30.731945 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.731920    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.833065 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.833005    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.927830 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:30.927799    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076cef5425bde9780a5fcdde6e01c168.slice/crio-6811a7ccc9ab0854118ba3f305735c69cf9515953dc022f35a710c4a42aa7e6d WatchSource:0}: Error finding container 6811a7ccc9ab0854118ba3f305735c69cf9515953dc022f35a710c4a42aa7e6d: Status 404 returned error can't find the container with id 6811a7ccc9ab0854118ba3f305735c69cf9515953dc022f35a710c4a42aa7e6d
Sep 18 20:20:30.928322 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:30.928300    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27b89508e8bec15507dad5bcf40fdba.slice/crio-739b414b9577e6e91516581f2686ecb304db4867b5c3f685d3de276bdad12440 WatchSource:0}: Error finding container 739b414b9577e6e91516581f2686ecb304db4867b5c3f685d3de276bdad12440: Status 404 returned error can't find the container with id 739b414b9577e6e91516581f2686ecb304db4867b5c3f685d3de276bdad12440
Sep 18 20:20:30.933245 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:30.933219    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:30.933371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:30.933263    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:20:31.034154 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.034122    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:31.135208 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.135155    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-180.ec2.internal\" not found"
Sep 18 20:20:31.187887 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.187866    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:31.223796 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.223765    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:31.236662 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.236644    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:20:31.237503 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.237490    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal"
Sep 18 20:20:31.247021 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.247004    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:20:31.493182 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.493061    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:31.666785 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.666753    2564 apiserver.go:52] "Watching apiserver"
Sep 18 20:20:31.679396 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.679366    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 20:20:31.680415 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.679944    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal","openshift-dns/node-resolver-n8wqs","openshift-image-registry/node-ca-2hj9w","openshift-multus/multus-hpt84","openshift-network-diagnostics/network-check-target-x5lgz","openshift-network-operator/iptables-alerter-9wlgj","openshift-ovn-kubernetes/ovnkube-node-7nqtg","kube-system/global-pull-secret-syncer-7q8s6","kube-system/konnectivity-agent-tx7jf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb","openshift-cluster-node-tuning-operator/tuned-tfrfz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal","openshift-multus/multus-additional-cni-plugins-q9hpx","openshift-multus/network-metrics-daemon-s4rsp"]
Sep 18 20:20:31.682938 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.682890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.685676 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.685440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.688474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.688046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.688474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.688466    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 20:20:31.688778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.688713    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vcv59\""
Sep 18 20:20:31.690641 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.689534    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.690641 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.689794    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 20:20:31.690641 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.689833    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 20:20:31.691498 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.691476    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 20:20:31.691710 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.691681    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 20:20:31.691804 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.691751    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.691979 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.691959    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dskct\""
Sep 18 20:20:31.692363 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.692344    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.692472 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.692451    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.693221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.691752    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 20:20:31.693221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.692923    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.693221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.692950    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.693221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.693004    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 20:20:31.694189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.693982    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mmwjw\""
Sep 18 20:20:31.694365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.694342    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 20:20:31.695758 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.695737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.699082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.699061    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 20:20:31.699296 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.699278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l49cp\""
Sep 18 20:20:31.699769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.699753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:31.700341 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.699966    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:31.702233 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.702208    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.704971 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.704953    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 20:20:31.705128 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.705114    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.705297 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.705282    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.705437 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.705421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cph4m\""
Sep 18 20:20:31.705606 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.705585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.707548 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.707528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.707645 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.707594    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:31.708186 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.708164    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 20:20:31.708602 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.708577    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 20:20:31.708767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.708749    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sm4r4\""
Sep 18 20:20:31.710616 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.710137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.713841 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.712805    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.713841 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.712931    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.713841 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.713235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sfhnx\""
Sep 18 20:20:31.713841 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.713352    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.715587 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.715567    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.716831 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.716334    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n98ss\""
Sep 18 20:20:31.716831 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.716554    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.716996 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.716849    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 20:20:31.717345 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.717214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.720619 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.720596    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.720731 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.720695    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:31.721927 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.721905    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tb9f9\""
Sep 18 20:20:31.722223 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.722011    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:20:31.722223 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.722086    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 20:20:31.723078 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.723059    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 20:20:31.724828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.724809    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 20:20:31.735825 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.735802    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:20:31.737675 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-hosts-file\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.737783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-systemd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.737783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-cnibin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.737783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-socket-dir-parent\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.737783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-device-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737797    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-var-lib-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovn-node-metrics-cert\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-cni-binary-copy\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffa8e5a9-4ee7-432e-a387-b9270556c166-host-slash\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.737992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.737979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-slash\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysconfig\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738102    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-socket-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.738174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm6lk\" (UniqueName: \"kubernetes.io/projected/66791213-4af8-4a60-9399-19e12eb2b352-kube-api-access-sm6lk\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738182    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-modprobe-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-conf\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-ovn\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-config\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-bin\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738303    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f45420e-b2a2-40d0-9180-d0a44bd128ef-konnectivity-ca\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.738353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-var-lib-kubelet\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-systemd-units\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-kubelet\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-multus-daemon-config\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.738578 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-registration-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6s68\" (UniqueName: \"kubernetes.io/projected/17ece2c5-d02f-4898-871f-313bb14f89ed-kube-api-access-k6s68\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738619    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66791213-4af8-4a60-9399-19e12eb2b352-serviceca\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f45420e-b2a2-40d0-9180-d0a44bd128ef-agent-certs\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-log-socket\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-netd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-multus\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqjph\" (UniqueName: \"kubernetes.io/projected/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-kube-api-access-zqjph\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-system-cni-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.738892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cnibin\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-system-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-hostroot\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-sys\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.738983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-tmp\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8v4q\" (UniqueName: \"kubernetes.io/projected/236aafa2-de48-4bf3-b41a-707da2d61501-kube-api-access-z8v4q\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66791213-4af8-4a60-9399-19e12eb2b352-host\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-systemd\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-kubelet\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-k8s-cni-cncf-io\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-multus-certs\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-lib-modules\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739325    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-netns\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-env-overrides\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.739443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-etc-kubernetes\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6w9d\" (UniqueName: \"kubernetes.io/projected/b290c323-1705-4d49-8190-e0868c57b6d1-kube-api-access-j6w9d\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88nh\" (UniqueName: \"kubernetes.io/projected/ffa8e5a9-4ee7-432e-a387-b9270556c166-kube-api-access-t88nh\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-tmp-dir\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-run\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-host\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-tuned\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-etc-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-script-lib\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739751    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-sys-fs\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-kubelet-config\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbmd\" (UniqueName: \"kubernetes.io/projected/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-kube-api-access-7cbmd\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-bin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffa8e5a9-4ee7-432e-a387-b9270556c166-iptables-alerter-script\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx56l\" (UniqueName: \"kubernetes.io/projected/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-kube-api-access-nx56l\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.740259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739953    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-netns\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.739976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-conf-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhlh2\" (UniqueName: \"kubernetes.io/projected/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-kube-api-access-rhlh2\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740026    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-dbus\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-node-log\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-kubernetes\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.741031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.740149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.776057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.775293    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7nzmv"
Sep 18 20:20:31.787880 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.787857    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7nzmv"
Sep 18 20:20:31.793434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.793381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal" event={"ID":"a27b89508e8bec15507dad5bcf40fdba","Type":"ContainerStarted","Data":"739b414b9577e6e91516581f2686ecb304db4867b5c3f685d3de276bdad12440"}
Sep 18 20:20:31.794753 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.794725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal" event={"ID":"076cef5425bde9780a5fcdde6e01c168","Type":"ContainerStarted","Data":"6811a7ccc9ab0854118ba3f305735c69cf9515953dc022f35a710c4a42aa7e6d"}
Sep 18 20:20:31.840903 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-slash\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysconfig\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-socket-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm6lk\" (UniqueName: \"kubernetes.io/projected/66791213-4af8-4a60-9399-19e12eb2b352-kube-api-access-sm6lk\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.840998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-modprobe-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-conf\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.841058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-ovn\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-config\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-bin\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f45420e-b2a2-40d0-9180-d0a44bd128ef-konnectivity-ca\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841187    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-var-lib-kubelet\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-systemd-units\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-conf\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-kubelet\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-ovn\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.841434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysconfig\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-systemd-units\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.841521    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.841600    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:32.34157684 +0000 UTC m=+4.690849217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-kubelet\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-bin\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-socket-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f45420e-b2a2-40d0-9180-d0a44bd128ef-konnectivity-ca\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-slash\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-var-lib-kubelet\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-modprobe-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-multus-daemon-config\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841950    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-registration-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.841998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6s68\" (UniqueName: \"kubernetes.io/projected/17ece2c5-d02f-4898-871f-313bb14f89ed-kube-api-access-k6s68\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.842002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-multus-daemon-config\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66791213-4af8-4a60-9399-19e12eb2b352-serviceca\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f45420e-b2a2-40d0-9180-d0a44bd128ef-agent-certs\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-log-socket\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842151    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-netd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-multus\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842202    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqjph\" (UniqueName: \"kubernetes.io/projected/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-kube-api-access-zqjph\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-system-cni-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cnibin\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-registration-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-system-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-config\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-hostroot\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-multus\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-sys\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.842839 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-tmp\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8v4q\" (UniqueName: \"kubernetes.io/projected/236aafa2-de48-4bf3-b41a-707da2d61501-kube-api-access-z8v4q\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66791213-4af8-4a60-9399-19e12eb2b352-serviceca\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-log-socket\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-cni-netd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66791213-4af8-4a60-9399-19e12eb2b352-host\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842593    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-system-cni-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-systemd\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842626    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-system-cni-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842632    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-kubelet\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-hostroot\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cnibin\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66791213-4af8-4a60-9399-19e12eb2b352-host\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-k8s-cni-cncf-io\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.843688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842735    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-sys\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-multus-certs\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-kubelet\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842781    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-lib-modules\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-multus-certs\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-systemd\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842880    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-lib-modules\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-k8s-cni-cncf-io\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-netns\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-host-run-netns\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-env-overrides\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.842996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-etc-kubernetes\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6w9d\" (UniqueName: \"kubernetes.io/projected/b290c323-1705-4d49-8190-e0868c57b6d1-kube-api-access-j6w9d\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88nh\" (UniqueName: \"kubernetes.io/projected/ffa8e5a9-4ee7-432e-a387-b9270556c166-kube-api-access-t88nh\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.844631 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-tmp-dir\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-run\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-host\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-tuned\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-etc-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-script-lib\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-sys-fs\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-kubelet-config\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbmd\" (UniqueName: \"kubernetes.io/projected/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-kube-api-access-7cbmd\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-bin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-etc-kubernetes\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffa8e5a9-4ee7-432e-a387-b9270556c166-iptables-alerter-script\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx56l\" (UniqueName: \"kubernetes.io/projected/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-kube-api-access-nx56l\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843413    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-sys-fs\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-netns\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-conf-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.845563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-etc-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhlh2\" (UniqueName: \"kubernetes.io/projected/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-kube-api-access-rhlh2\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-dbus\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-node-log\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-env-overrides\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-kubernetes\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-hosts-file\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-systemd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-cnibin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843746    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-socket-dir-parent\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-device-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-var-lib-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovn-node-metrics-cert\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.847331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-cni-binary-copy\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-tmp-dir\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffa8e5a9-4ee7-432e-a387-b9270556c166-host-slash\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.843953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-run\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovnkube-script-lib\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.844051    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844060    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-kubernetes\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844099    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-device-dir\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.844109    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:20:32.344091984 +0000 UTC m=+4.693364360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-kubelet-config\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-run-systemd\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b290c323-1705-4d49-8190-e0868c57b6d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-cnibin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-hosts-file\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-socket-dir-parent\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.848214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-var-lib-openvswitch\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-host\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844361    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-var-lib-cni-bin\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffa8e5a9-4ee7-432e-a387-b9270556c166-host-slash\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-sysctl-d\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-multus-conf-dir\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/236aafa2-de48-4bf3-b41a-707da2d61501-cni-binary-copy\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.844853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/392944fa-d16b-411b-86a8-62c6985ee1a5-dbus\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.845016    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/236aafa2-de48-4bf3-b41a-707da2d61501-host-run-netns\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.845051    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffa8e5a9-4ee7-432e-a387-b9270556c166-iptables-alerter-script\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.845062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-node-log\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.845356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.846485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-tmp\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.846912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f45420e-b2a2-40d0-9180-d0a44bd128ef-agent-certs\") pod \"konnectivity-agent-tx7jf\" (UID: \"4f45420e-b2a2-40d0-9180-d0a44bd128ef\") " pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.847094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-etc-tuned\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.849024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.847274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-ovn-node-metrics-cert\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.850937 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.850915    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:31.851031 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.850942    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:31.851031 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.850956    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:31.851196 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:31.851082    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:32.351064587 +0000 UTC m=+4.700336984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:31.852575 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.852550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm6lk\" (UniqueName: \"kubernetes.io/projected/66791213-4af8-4a60-9399-19e12eb2b352-kube-api-access-sm6lk\") pod \"node-ca-2hj9w\" (UID: \"66791213-4af8-4a60-9399-19e12eb2b352\") " pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:31.853563 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.853534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6s68\" (UniqueName: \"kubernetes.io/projected/17ece2c5-d02f-4898-871f-313bb14f89ed-kube-api-access-k6s68\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:31.856386 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.856353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88nh\" (UniqueName: \"kubernetes.io/projected/ffa8e5a9-4ee7-432e-a387-b9270556c166-kube-api-access-t88nh\") pod \"iptables-alerter-9wlgj\" (UID: \"ffa8e5a9-4ee7-432e-a387-b9270556c166\") " pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:31.856951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.856923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhlh2\" (UniqueName: \"kubernetes.io/projected/7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5-kube-api-access-rhlh2\") pod \"tuned-tfrfz\" (UID: \"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5\") " pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:31.857452 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.857418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8v4q\" (UniqueName: \"kubernetes.io/projected/236aafa2-de48-4bf3-b41a-707da2d61501-kube-api-access-z8v4q\") pod \"multus-hpt84\" (UID: \"236aafa2-de48-4bf3-b41a-707da2d61501\") " pod="openshift-multus/multus-hpt84"
Sep 18 20:20:31.857688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.857646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqjph\" (UniqueName: \"kubernetes.io/projected/3976f7a8-ad32-42e8-ac8b-7b71abafe3c7-kube-api-access-zqjph\") pod \"ovnkube-node-7nqtg\" (UID: \"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:31.857805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.857783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbmd\" (UniqueName: \"kubernetes.io/projected/8b96350d-9e01-4ec8-9ac7-3f4929a6eac4-kube-api-access-7cbmd\") pod \"multus-additional-cni-plugins-q9hpx\" (UID: \"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4\") " pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:31.859128 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.859078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx56l\" (UniqueName: \"kubernetes.io/projected/7e641a0c-60b0-40a9-bf65-7fea23ffdd29-kube-api-access-nx56l\") pod \"node-resolver-n8wqs\" (UID: \"7e641a0c-60b0-40a9-bf65-7fea23ffdd29\") " pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:31.859216 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:31.859171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6w9d\" (UniqueName: \"kubernetes.io/projected/b290c323-1705-4d49-8190-e0868c57b6d1-kube-api-access-j6w9d\") pod \"aws-ebs-csi-driver-node-hqlcb\" (UID: \"b290c323-1705-4d49-8190-e0868c57b6d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:32.002809 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.002777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:32.013272 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.013215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q9hpx"
Sep 18 20:20:32.025261 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.025238    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2hj9w"
Sep 18 20:20:32.036406 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.036371    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hpt84"
Sep 18 20:20:32.045454 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.045424    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9wlgj"
Sep 18 20:20:32.060623 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.060602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:32.069844 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.069818    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8wqs"
Sep 18 20:20:32.079770 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.079753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb"
Sep 18 20:20:32.086734 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.086709    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tfrfz"
Sep 18 20:20:32.088296 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.088277    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:32.347504 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.347431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:32.347504 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.347472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:32.347743 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.347599    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:32.347743 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.347607    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:32.347743 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.347659    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:33.347641881 +0000 UTC m=+5.696914275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:32.347743 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.347686    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:20:33.347666754 +0000 UTC m=+5.696939146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:32.448566 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.448530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:32.448771 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.448749    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:32.448861 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.448783    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:32.448861 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.448798    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:32.448964 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:32.448866    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:33.448845539 +0000 UTC m=+5.798117929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:32.507213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.507187    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:32.791955 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.791887    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 20:15:31 +0000 UTC" deadline="2028-06-08 17:54:53.575855974 +0000 UTC"
Sep 18 20:20:32.791955 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:32.791915    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15093h34m20.783943982s"
Sep 18 20:20:33.038685 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.038655    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3976f7a8_ad32_42e8_ac8b_7b71abafe3c7.slice/crio-0d5d93f21413557605f8b29980b177023bfb86ee510800f75d25ef8d559c60d1 WatchSource:0}: Error finding container 0d5d93f21413557605f8b29980b177023bfb86ee510800f75d25ef8d559c60d1: Status 404 returned error can't find the container with id 0d5d93f21413557605f8b29980b177023bfb86ee510800f75d25ef8d559c60d1
Sep 18 20:20:33.041767 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.041464    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb290c323_1705_4d49_8190_e0868c57b6d1.slice/crio-303497ac4d436631b0ada8e1c4996493d74bca16130352e6d2479efda3e15f24 WatchSource:0}: Error finding container 303497ac4d436631b0ada8e1c4996493d74bca16130352e6d2479efda3e15f24: Status 404 returned error can't find the container with id 303497ac4d436631b0ada8e1c4996493d74bca16130352e6d2479efda3e15f24
Sep 18 20:20:33.043931 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.043899    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f45420e_b2a2_40d0_9180_d0a44bd128ef.slice/crio-53a7444b2acf5b6f5090b1d572ed2d719eeb80447aed6d6f3826936ce961e69b WatchSource:0}: Error finding container 53a7444b2acf5b6f5090b1d572ed2d719eeb80447aed6d6f3826936ce961e69b: Status 404 returned error can't find the container with id 53a7444b2acf5b6f5090b1d572ed2d719eeb80447aed6d6f3826936ce961e69b
Sep 18 20:20:33.045240 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.044777    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66791213_4af8_4a60_9399_19e12eb2b352.slice/crio-f027d02915e2fc994dacfd11cd12fbf7fb2321a41220b278fd68983dd478f771 WatchSource:0}: Error finding container f027d02915e2fc994dacfd11cd12fbf7fb2321a41220b278fd68983dd478f771: Status 404 returned error can't find the container with id f027d02915e2fc994dacfd11cd12fbf7fb2321a41220b278fd68983dd478f771
Sep 18 20:20:33.045523 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.045445    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3f77f9_109d_4a66_8552_4d4bb3a3f6d5.slice/crio-b2a51f94ccc70ff0329cadb8580dc8ddad833d13c6884fe32763ce29b883c9a1 WatchSource:0}: Error finding container b2a51f94ccc70ff0329cadb8580dc8ddad833d13c6884fe32763ce29b883c9a1: Status 404 returned error can't find the container with id b2a51f94ccc70ff0329cadb8580dc8ddad833d13c6884fe32763ce29b883c9a1
Sep 18 20:20:33.046712 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.046691    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e641a0c_60b0_40a9_bf65_7fea23ffdd29.slice/crio-1261ee31304ae578048a43d4c631bf5a840772f86768d8072549bfcffd93379b WatchSource:0}: Error finding container 1261ee31304ae578048a43d4c631bf5a840772f86768d8072549bfcffd93379b: Status 404 returned error can't find the container with id 1261ee31304ae578048a43d4c631bf5a840772f86768d8072549bfcffd93379b
Sep 18 20:20:33.047834 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.047516    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa8e5a9_4ee7_432e_a387_b9270556c166.slice/crio-ce99aa6f04ba46f9d419dbd1f1b0daf202623356dc4379ffa04c4fbbf550b4c4 WatchSource:0}: Error finding container ce99aa6f04ba46f9d419dbd1f1b0daf202623356dc4379ffa04c4fbbf550b4c4: Status 404 returned error can't find the container with id ce99aa6f04ba46f9d419dbd1f1b0daf202623356dc4379ffa04c4fbbf550b4c4
Sep 18 20:20:33.051481 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:20:33.050247    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236aafa2_de48_4bf3_b41a_707da2d61501.slice/crio-716ee301dc75598765a1f2bdb4199c027319802211dbc6039ff4896dad9d4f45 WatchSource:0}: Error finding container 716ee301dc75598765a1f2bdb4199c027319802211dbc6039ff4896dad9d4f45: Status 404 returned error can't find the container with id 716ee301dc75598765a1f2bdb4199c027319802211dbc6039ff4896dad9d4f45
Sep 18 20:20:33.355472 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.355311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:33.355584 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.355526    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:33.355584 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.355451    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:33.355704 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.355609    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:33.355704 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.355617    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:35.355595602 +0000 UTC m=+7.704867975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:33.355704 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.355665    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:20:35.355653943 +0000 UTC m=+7.704926314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:33.459310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.456743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:33.459310 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.456925    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:33.459310 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.456953    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:33.459310 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.456968    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:33.459310 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.457028    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:35.45700808 +0000 UTC m=+7.806280473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:33.790897 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.790821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:33.791040 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.790944    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:33.791375 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.791346    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:33.791477 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.791456    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:33.791545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.791523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:33.791603 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:33.791592    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:33.792264 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.792216    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 20:15:31 +0000 UTC" deadline="2028-05-19 02:13:28.627972377 +0000 UTC"
Sep 18 20:20:33.792624 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.792266    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14597h52m54.835710687s"
Sep 18 20:20:33.819095 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.819068    2564 generic.go:356] "Generic (PLEG): container finished" podID="a27b89508e8bec15507dad5bcf40fdba" containerID="19bf34e944b2a194fe825b5ad911ce80231f14da9eb98e844b0418a64fef469e" exitCode=0
Sep 18 20:20:33.819207 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.819160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal" event={"ID":"a27b89508e8bec15507dad5bcf40fdba","Type":"ContainerDied","Data":"19bf34e944b2a194fe825b5ad911ce80231f14da9eb98e844b0418a64fef469e"}
Sep 18 20:20:33.830199 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.830179    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerStarted","Data":"364069e518800431914f6d77f7b4bbe862ae910c9cc79ef34d9f0c0cd076b95f"}
Sep 18 20:20:33.833468 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.833405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9wlgj" event={"ID":"ffa8e5a9-4ee7-432e-a387-b9270556c166","Type":"ContainerStarted","Data":"ce99aa6f04ba46f9d419dbd1f1b0daf202623356dc4379ffa04c4fbbf550b4c4"}
Sep 18 20:20:33.843411 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.843367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2hj9w" event={"ID":"66791213-4af8-4a60-9399-19e12eb2b352","Type":"ContainerStarted","Data":"f027d02915e2fc994dacfd11cd12fbf7fb2321a41220b278fd68983dd478f771"}
Sep 18 20:20:33.859856 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.859827    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal" event={"ID":"076cef5425bde9780a5fcdde6e01c168","Type":"ContainerStarted","Data":"ce0ccadc557167b61ea6e57a4de1936652b755360b50957f34da13407cd3c028"}
Sep 18 20:20:33.871949 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.871838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hpt84" event={"ID":"236aafa2-de48-4bf3-b41a-707da2d61501","Type":"ContainerStarted","Data":"716ee301dc75598765a1f2bdb4199c027319802211dbc6039ff4896dad9d4f45"}
Sep 18 20:20:33.878365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.878180    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-180.ec2.internal" podStartSLOduration=2.877658512 podStartE2EDuration="2.877658512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:33.876607865 +0000 UTC m=+6.225880285" watchObservedRunningTime="2026-09-18 20:20:33.877658512 +0000 UTC m=+6.226930906"
Sep 18 20:20:33.884733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.884472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8wqs" event={"ID":"7e641a0c-60b0-40a9-bf65-7fea23ffdd29","Type":"ContainerStarted","Data":"1261ee31304ae578048a43d4c631bf5a840772f86768d8072549bfcffd93379b"}
Sep 18 20:20:33.888895 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.888872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tfrfz" event={"ID":"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5","Type":"ContainerStarted","Data":"b2a51f94ccc70ff0329cadb8580dc8ddad833d13c6884fe32763ce29b883c9a1"}
Sep 18 20:20:33.907079 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.905484    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tx7jf" event={"ID":"4f45420e-b2a2-40d0-9180-d0a44bd128ef","Type":"ContainerStarted","Data":"53a7444b2acf5b6f5090b1d572ed2d719eeb80447aed6d6f3826936ce961e69b"}
Sep 18 20:20:33.921840 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.921803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb" event={"ID":"b290c323-1705-4d49-8190-e0868c57b6d1","Type":"ContainerStarted","Data":"303497ac4d436631b0ada8e1c4996493d74bca16130352e6d2479efda3e15f24"}
Sep 18 20:20:33.939028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:33.939001    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"0d5d93f21413557605f8b29980b177023bfb86ee510800f75d25ef8d559c60d1"}
Sep 18 20:20:34.962673 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:34.962620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal" event={"ID":"a27b89508e8bec15507dad5bcf40fdba","Type":"ContainerStarted","Data":"86e435712de80716cd5072986e9e8f03078c472e43fc081af75299c5561dd0b7"}
Sep 18 20:20:35.373979 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.373894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:35.374144 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.373989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:35.374144 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.374094    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:35.374257 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.374175    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:20:39.374154582 +0000 UTC m=+11.723426973 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:35.374257 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.374100    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:35.374257 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.374211    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:39.374200836 +0000 UTC m=+11.723473221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:35.475285 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.475247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:35.475496 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.475478    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:35.475561 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.475504    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:35.475561 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.475518    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:35.475686 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.475577    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:39.475559167 +0000 UTC m=+11.824831543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:35.787800 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.787725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:35.787955 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.787858    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:35.788274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.788252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:35.788385 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.788365    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:35.788455 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:35.788421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:35.788505 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:35.788494    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:37.788365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:37.788331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:37.788837 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:37.788461    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:37.788837 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:37.788496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:37.788837 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:37.788571    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:37.788837 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:37.788688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:37.788837 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:37.788773    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:39.408642 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.408567    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:39.408642 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.408621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:39.409156 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.408752    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:39.409156 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.408831    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:20:47.408809925 +0000 UTC m=+19.758082310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:39.409156 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.408762    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:39.409156 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.408902    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:47.408883768 +0000 UTC m=+19.758156141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:39.509931 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.509890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:39.510124 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.510102    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:39.510203 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.510132    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:39.510203 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.510148    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:39.510373 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.510215    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:47.510199182 +0000 UTC m=+19.859471574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:39.789786 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.789664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:39.789937 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.789834    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:39.790042 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.790019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:39.790158 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:39.790072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:39.790216 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.790162    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:39.790307 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:39.790268    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:41.788096 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:41.788064    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:41.788543 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:41.788067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:41.788543 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:41.788177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:41.788543 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:41.788217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:41.788543 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:41.788331    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:41.788543 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:41.788413    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:43.788215 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:43.788180    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:43.788778 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:43.788306    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:43.788778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:43.788344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:43.788778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:43.788358    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:43.788778 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:43.788438    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:43.788778 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:43.788540    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:45.788600 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:45.788560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:45.788972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:45.788620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:45.788972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:45.788566    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:45.788972 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:45.788717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:45.788972 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:45.788783    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:45.788972 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:45.788886    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:47.464021 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.463989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:47.464492 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.464038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:47.464492 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.464155    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:47.464492 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.464161    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:47.464492 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.464218    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:21:03.464201475 +0000 UTC m=+35.813473859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:47.464492 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.464284    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:03.464263389 +0000 UTC m=+35.813535785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:47.565076 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.565042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:47.565259 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.565211    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:47.565259 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.565236    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:47.565259 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.565249    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:47.565406 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.565311    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:03.565293835 +0000 UTC m=+35.914566228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:47.788082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.788002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:47.788082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.788040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:47.788274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:47.788011    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:47.788274 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.788134    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:47.788274 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.788227    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:47.788404 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:47.788279    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:49.790231 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:49.789716    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:49.790722 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:49.790623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:49.790797 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:49.790766    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:49.790885 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:49.790858    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:49.790977 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:49.790947    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:49.791306 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:49.791283    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:50.997269 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:50.997243    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="47b21caa7420671b028e5194bf99955b237f0eeb89ea9c1a8cd4c6683b9f6c8b" exitCode=0
Sep 18 20:20:50.997726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:50.997310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"47b21caa7420671b028e5194bf99955b237f0eeb89ea9c1a8cd4c6683b9f6c8b"}
Sep 18 20:20:50.998956 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:50.998926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2hj9w" event={"ID":"66791213-4af8-4a60-9399-19e12eb2b352","Type":"ContainerStarted","Data":"e345a2ac034a3d2dfb1866c14ce3ee6a7631cae2c92961e5af048b25cc28be8f"}
Sep 18 20:20:51.000588 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.000563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hpt84" event={"ID":"236aafa2-de48-4bf3-b41a-707da2d61501","Type":"ContainerStarted","Data":"412d52a64c6f4cd15a4741af1ca46be6f625fa6e10dd5b9b55facad0db10b409"}
Sep 18 20:20:51.001984 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.001955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8wqs" event={"ID":"7e641a0c-60b0-40a9-bf65-7fea23ffdd29","Type":"ContainerStarted","Data":"19bf11105701a97a169bbd38c7cde26b3288840dc15ed5e785a5992dcd6a0283"}
Sep 18 20:20:51.003601 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.003578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tfrfz" event={"ID":"7e3f77f9-109d-4a66-8552-4d4bb3a3f6d5","Type":"ContainerStarted","Data":"26feb7ede2435152bba91d3507a42a4c80bc795cc4b21c94115035aa6fc8a521"}
Sep 18 20:20:51.005108 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.005088    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tx7jf" event={"ID":"4f45420e-b2a2-40d0-9180-d0a44bd128ef","Type":"ContainerStarted","Data":"77ab6d46b5c894dc8b5d68387042f041edbd4af5128f87ff53ac9276f23cfc19"}
Sep 18 20:20:51.006689 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.006667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb" event={"ID":"b290c323-1705-4d49-8190-e0868c57b6d1","Type":"ContainerStarted","Data":"8a86b442e2c018ed321f57ad7f952016b93f18f26bb3dce7e87d3957a586ec4f"}
Sep 18 20:20:51.008594 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.008576    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:20:51.008979 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.008958    2564 generic.go:356] "Generic (PLEG): container finished" podID="3976f7a8-ad32-42e8-ac8b-7b71abafe3c7" containerID="70b453d7a2b5956430b87e91fc46b1f9b3c2716785c37d99553d0e3f4ea2cfc2" exitCode=1
Sep 18 20:20:51.009053 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.008989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"556bad9ce4f17667d99c4e4d8b6b2847531e321066d203a7fa2468b878b76b5e"}
Sep 18 20:20:51.009053 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.009041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerDied","Data":"70b453d7a2b5956430b87e91fc46b1f9b3c2716785c37d99553d0e3f4ea2cfc2"}
Sep 18 20:20:51.009119 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.009076    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"c880e0671088be80173b2f8e51b19e045f368b3996f1288172b664a23963c8cc"}
Sep 18 20:20:51.020700 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.020664    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-180.ec2.internal" podStartSLOduration=20.020652192 podStartE2EDuration="20.020652192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:34.979799971 +0000 UTC m=+7.329072380" watchObservedRunningTime="2026-09-18 20:20:51.020652192 +0000 UTC m=+23.369924576"
Sep 18 20:20:51.038269 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.038227    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hpt84" podStartSLOduration=3.655108328 podStartE2EDuration="21.038213166s" totalImagesPullingTime="17.383104838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:51.03778782 +0000 UTC m=+23.387060235" watchObservedRunningTime="2026-09-18 20:20:51.038213166 +0000 UTC m=+23.387485567"
Sep 18 20:20:51.052840 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.052781    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2hj9w" podStartSLOduration=3.677573265 podStartE2EDuration="21.052767363s" totalImagesPullingTime="17.375194098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:51.052578193 +0000 UTC m=+23.401850609" watchObservedRunningTime="2026-09-18 20:20:51.052767363 +0000 UTC m=+23.402039802"
Sep 18 20:20:51.067380 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.067344    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tx7jf" podStartSLOduration=11.285564721 podStartE2EDuration="21.067330167s" totalImagesPullingTime="9.781765446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:51.067193593 +0000 UTC m=+23.416465985" watchObservedRunningTime="2026-09-18 20:20:51.067330167 +0000 UTC m=+23.416602557"
Sep 18 20:20:51.084141 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.084081    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n8wqs" podStartSLOduration=3.710004234 podStartE2EDuration="21.084062508s" totalImagesPullingTime="17.374058274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:51.083851602 +0000 UTC m=+23.433123995" watchObservedRunningTime="2026-09-18 20:20:51.084062508 +0000 UTC m=+23.433334902"
Sep 18 20:20:51.104797 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.104493    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tfrfz" podStartSLOduration=3.727185343 podStartE2EDuration="21.104473896s" totalImagesPullingTime="17.377288553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:51.104024609 +0000 UTC m=+23.453297028" watchObservedRunningTime="2026-09-18 20:20:51.104473896 +0000 UTC m=+23.453746292"
Sep 18 20:20:51.788440 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.788230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:51.788440 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.788262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:51.788440 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:51.788356    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:51.788738 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:51.788454    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:51.788738 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.788521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:51.788738 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:51.788613    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:51.794599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:51.794547    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 20:20:52.013323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.013286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb" event={"ID":"b290c323-1705-4d49-8190-e0868c57b6d1","Type":"ContainerStarted","Data":"cc84c572c7c5d093380d2f38a7c80c5b341c95ea493272d6add56a5bf9a74f03"}
Sep 18 20:20:52.016577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.016558    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:20:52.017019 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.016988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"3ecfdc8d33f431674dcd47aa5f9f4051c25898803ce407b9c9187aa7cf385d14"}
Sep 18 20:20:52.017116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.017056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"fca4c06f068615a52b6f8c3163f34dfd8b52b8094f84bf4d46321cc3fab95005"}
Sep 18 20:20:52.017116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.017097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"eb461356544156f7fd5e41c1c2687e28a41a3b46e8925f8972c66c58ad1eab08"}
Sep 18 20:20:52.018583 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.018519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9wlgj" event={"ID":"ffa8e5a9-4ee7-432e-a387-b9270556c166","Type":"ContainerStarted","Data":"bcc5e5ccccbfbf365d50d187b8ef23594e44090eb141cf21a6e8652bf4b0d46a"}
Sep 18 20:20:52.036951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.036857    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9wlgj" podStartSLOduration=4.694380426 podStartE2EDuration="22.036845002s" totalImagesPullingTime="17.342464576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:52.036762425 +0000 UTC m=+24.386034837" watchObservedRunningTime="2026-09-18 20:20:52.036845002 +0000 UTC m=+24.386117394"
Sep 18 20:20:52.793098 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.792995    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T20:20:51.79456409Z","UUID":"00d00288-f1f2-4a42-b952-7f897c3d5021","Handler":null,"Name":"","Endpoint":""}
Sep 18 20:20:52.794548 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.794528    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 20:20:52.794678 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:52.794557    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 20:20:53.023167 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:53.023141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb" event={"ID":"b290c323-1705-4d49-8190-e0868c57b6d1","Type":"ContainerStarted","Data":"69bb335cf9f0629728e7ebaea9ee9ad7539c0d613d82568ba10d554ffa88ffbe"}
Sep 18 20:20:53.788030 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:53.787954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:53.788173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:53.787965    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:53.788173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:53.788081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:53.788173 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:53.788080    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:53.788337 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:53.788170    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:53.788337 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:53.788247    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:54.029705 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:54.029673    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:20:54.030220 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:54.030182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"b65e162536c2b74a16a28314faa5c04b6de4f3511b38271137348314291e3a8a"}
Sep 18 20:20:54.578474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:54.578307    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:54.578940 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:54.578914    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:54.597162 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:54.597109    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqlcb" podStartSLOduration=5.200584995 podStartE2EDuration="24.597094331s" totalImagesPullingTime="19.396509336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:53.044885251 +0000 UTC m=+25.394157671" watchObservedRunningTime="2026-09-18 20:20:54.597094331 +0000 UTC m=+26.946366723"
Sep 18 20:20:55.032415 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:55.032346    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:55.032863 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:55.032841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tx7jf"
Sep 18 20:20:55.791702 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:55.791522    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:55.791871 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:55.791574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:55.791871 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:55.791772    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:55.791871 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:55.791573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:55.791871 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:55.791839    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:55.792047 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:55.791961    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:56.038052 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.037956    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:20:56.038721 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.038318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"8a5c3b7ae1044deae7ce57d5c86b3e610bec340e3e998f4918f749b85b85ca55"}
Sep 18 20:20:56.038721 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.038542    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:56.038721 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.038624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:56.038721 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.038692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:56.038894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.038824    2564 scope.go:117] "RemoveContainer" containerID="70b453d7a2b5956430b87e91fc46b1f9b3c2716785c37d99553d0e3f4ea2cfc2"
Sep 18 20:20:56.040683 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.040659    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="d18fe3527bd021e2e3340595b4d04f15d54f5666de4bceb5fd4772597dab3197" exitCode=0
Sep 18 20:20:56.040798 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.040755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"d18fe3527bd021e2e3340595b4d04f15d54f5666de4bceb5fd4772597dab3197"}
Sep 18 20:20:56.054868 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.054847    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:56.054960 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:56.054949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:20:57.046213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.046186    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:20:57.046675 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.046624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" event={"ID":"3976f7a8-ad32-42e8-ac8b-7b71abafe3c7","Type":"ContainerStarted","Data":"2693395ceddc9bcbb2c19791615b9c278dc273128475f05dae8202884989dde4"}
Sep 18 20:20:57.075567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.075517    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg" podStartSLOduration=9.652203851 podStartE2EDuration="27.07550302s" totalImagesPullingTime="17.423299169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:57.074038871 +0000 UTC m=+29.423311292" watchObservedRunningTime="2026-09-18 20:20:57.07550302 +0000 UTC m=+29.424775414"
Sep 18 20:20:57.510594 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.510558    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q8s6"]
Sep 18 20:20:57.510767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.510715    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:57.510849 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:57.510825    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:57.514291 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.514261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x5lgz"]
Sep 18 20:20:57.514408 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.514363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:57.514474 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:57.514448    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:20:57.514866 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.514845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s4rsp"]
Sep 18 20:20:57.514969 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:57.514955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:57.515076 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:57.515058    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:58.050624 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:58.050595    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="3c69f9721ba57c4aae6f55284f961d3297c1f01cecaa64226fc74445d9a7d095" exitCode=0
Sep 18 20:20:58.051008 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:58.050674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"3c69f9721ba57c4aae6f55284f961d3297c1f01cecaa64226fc74445d9a7d095"}
Sep 18 20:20:58.787926 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:58.787891    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:20:58.788080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:58.787898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:20:58.788080 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:58.788001    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:20:58.788209 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:58.788078    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:20:59.789331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:20:59.789302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:20:59.789681 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:20:59.789385    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:21:00.057706 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:00.057614    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="a424b7449481550f2a1025aa449f761558a9b4591e1344b265b72e1467bc71b6" exitCode=0
Sep 18 20:21:00.057706 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:00.057656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"a424b7449481550f2a1025aa449f761558a9b4591e1344b265b72e1467bc71b6"}
Sep 18 20:21:00.788169 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:00.788143    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:00.788347 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:00.788260    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:21:00.788347 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:00.788299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:00.788457 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:00.788390    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:21:01.788183 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:01.788108    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:01.788877 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:01.788240    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5lgz" podUID="8cabf748-848f-435c-af08-77e33aa4bd09"
Sep 18 20:21:02.788075 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:02.788048    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:02.788273 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:02.788053    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:02.788273 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:02.788167    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q8s6" podUID="392944fa-d16b-411b-86a8-62c6985ee1a5"
Sep 18 20:21:02.788273 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:02.788235    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4rsp" podUID="17ece2c5-d02f-4898-871f-313bb14f89ed"
Sep 18 20:21:03.486173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.486142    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:03.486423 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.486196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:03.486423 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.486324    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:21:03.486423 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.486326    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:21:03.486423 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.486376    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret podName:392944fa-d16b-411b-86a8-62c6985ee1a5 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:35.486362888 +0000 UTC m=+67.835635259 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret") pod "global-pull-secret-syncer-7q8s6" (UID: "392944fa-d16b-411b-86a8-62c6985ee1a5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:21:03.486423 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.486390    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs podName:17ece2c5-d02f-4898-871f-313bb14f89ed nodeName:}" failed. No retries permitted until 2026-09-18 20:21:35.486383764 +0000 UTC m=+67.835656135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs") pod "network-metrics-daemon-s4rsp" (UID: "17ece2c5-d02f-4898-871f-313bb14f89ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:21:03.585092 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.585065    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-180.ec2.internal" event="NodeReady"
Sep 18 20:21:03.585275 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.585229    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 20:21:03.586960 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.586926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:03.587136 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.587114    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:21:03.587224 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.587173    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:21:03.587224 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.587190    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7zg9d for pod openshift-network-diagnostics/network-check-target-x5lgz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:21:03.587291 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.587237    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d podName:8cabf748-848f-435c-af08-77e33aa4bd09 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:35.587224246 +0000 UTC m=+67.936496617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zg9d" (UniqueName: "kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d") pod "network-check-target-x5lgz" (UID: "8cabf748-848f-435c-af08-77e33aa4bd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:21:03.648064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.648027    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7ddcx"]
Sep 18 20:21:03.673103 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.673074    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mrj7p"]
Sep 18 20:21:03.673317 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.673297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.676760 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.676732    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jl6dr\""
Sep 18 20:21:03.676760 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.676737    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 20:21:03.676931 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.676782    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 20:21:03.676931 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.676745    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 20:21:03.693606 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.693525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ddcx"]
Sep 18 20:21:03.693735 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.693609    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.693735 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.693652    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mrj7p"]
Sep 18 20:21:03.698190 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.698168    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmtgt\""
Sep 18 20:21:03.698490 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.698471    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 20:21:03.698948 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.698930    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 20:21:03.788618 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788582    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-config-volume\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-cert\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-tmp-dir\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhcb7\" (UniqueName: \"kubernetes.io/projected/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-kube-api-access-lhcb7\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.789213 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.788994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dxtf\" (UniqueName: \"kubernetes.io/projected/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-kube-api-access-8dxtf\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.793111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.792928    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:21:03.793111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.793025    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmjzr\""
Sep 18 20:21:03.793749 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.793725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:21:03.890343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.890343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-tmp-dir\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.890343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhcb7\" (UniqueName: \"kubernetes.io/projected/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-kube-api-access-lhcb7\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.890665 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dxtf\" (UniqueName: \"kubernetes.io/projected/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-kube-api-access-8dxtf\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.890665 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.890444    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:03.890665 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-config-volume\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.890665 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-cert\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.890665 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:03.890516    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls podName:bc8aa2d9-ec75-4b25-9d02-e31a55b59338 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:04.390495147 +0000 UTC m=+36.739767537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls") pod "dns-default-mrj7p" (UID: "bc8aa2d9-ec75-4b25-9d02-e31a55b59338") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:03.890968 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.890766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-tmp-dir\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.891136 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.891115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-config-volume\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.894820 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.894799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-cert\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.899186 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.899161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dxtf\" (UniqueName: \"kubernetes.io/projected/cb5ab505-4689-4946-bb53-1e5c4fffe7a9-kube-api-access-8dxtf\") pod \"ingress-canary-7ddcx\" (UID: \"cb5ab505-4689-4946-bb53-1e5c4fffe7a9\") " pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:03.899297 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.899208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhcb7\" (UniqueName: \"kubernetes.io/projected/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-kube-api-access-lhcb7\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:03.984434 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:03.984391    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ddcx"
Sep 18 20:21:04.035203 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.035166    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"]
Sep 18 20:21:04.048128 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.048097    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"
Sep 18 20:21:04.049278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.049255    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"]
Sep 18 20:21:04.050907 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.050883    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4w5tf\""
Sep 18 20:21:04.193684 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.193521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd9p4\" (UniqueName: \"kubernetes.io/projected/c25929c2-99dc-4114-be12-f46793185392-kube-api-access-jd9p4\") pod \"network-check-source-7cbc45fddc-tgmm5\" (UID: \"c25929c2-99dc-4114-be12-f46793185392\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"
Sep 18 20:21:04.294910 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.294873    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd9p4\" (UniqueName: \"kubernetes.io/projected/c25929c2-99dc-4114-be12-f46793185392-kube-api-access-jd9p4\") pod \"network-check-source-7cbc45fddc-tgmm5\" (UID: \"c25929c2-99dc-4114-be12-f46793185392\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"
Sep 18 20:21:04.303588 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.303567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd9p4\" (UniqueName: \"kubernetes.io/projected/c25929c2-99dc-4114-be12-f46793185392-kube-api-access-jd9p4\") pod \"network-check-source-7cbc45fddc-tgmm5\" (UID: \"c25929c2-99dc-4114-be12-f46793185392\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"
Sep 18 20:21:04.358480 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.358451    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"
Sep 18 20:21:04.396148 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.396113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:04.396260 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:04.396239    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:04.396316 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:04.396300    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls podName:bc8aa2d9-ec75-4b25-9d02-e31a55b59338 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:05.396286627 +0000 UTC m=+37.745558999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls") pod "dns-default-mrj7p" (UID: "bc8aa2d9-ec75-4b25-9d02-e31a55b59338") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:04.398407 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.398369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ddcx"]
Sep 18 20:21:04.412779 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:04.412750    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5ab505_4689_4946_bb53_1e5c4fffe7a9.slice/crio-9a2e8b08b206a51a8c6005b17493902e29710c505a2aa384879a584f3d688c47 WatchSource:0}: Error finding container 9a2e8b08b206a51a8c6005b17493902e29710c505a2aa384879a584f3d688c47: Status 404 returned error can't find the container with id 9a2e8b08b206a51a8c6005b17493902e29710c505a2aa384879a584f3d688c47
Sep 18 20:21:04.691776 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.691706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5"]
Sep 18 20:21:04.717477 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:04.717448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25929c2_99dc_4114_be12_f46793185392.slice/crio-b5331b5dfcd72f098581d2d3e35098b06101f391b76a572d0994f51ef135e3ff WatchSource:0}: Error finding container b5331b5dfcd72f098581d2d3e35098b06101f391b76a572d0994f51ef135e3ff: Status 404 returned error can't find the container with id b5331b5dfcd72f098581d2d3e35098b06101f391b76a572d0994f51ef135e3ff
Sep 18 20:21:04.787660 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.787618    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:04.787823 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.787619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:04.790482 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.790463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:21:04.790850 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.790463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:21:04.790850 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:04.790506    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27fhx\""
Sep 18 20:21:05.076513 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:05.076436    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="c1ab34a123fea8a53fc3c3a88ce934b8b9a6f5e403bc7f7789950625e2eda034" exitCode=0
Sep 18 20:21:05.076703 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:05.076506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"c1ab34a123fea8a53fc3c3a88ce934b8b9a6f5e403bc7f7789950625e2eda034"}
Sep 18 20:21:05.078074 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:05.078049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ddcx" event={"ID":"cb5ab505-4689-4946-bb53-1e5c4fffe7a9","Type":"ContainerStarted","Data":"9a2e8b08b206a51a8c6005b17493902e29710c505a2aa384879a584f3d688c47"}
Sep 18 20:21:05.079298 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:05.079272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5" event={"ID":"c25929c2-99dc-4114-be12-f46793185392","Type":"ContainerStarted","Data":"b5331b5dfcd72f098581d2d3e35098b06101f391b76a572d0994f51ef135e3ff"}
Sep 18 20:21:05.402688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:05.402591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:05.402924 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:05.402758    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:05.402924 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:05.402829    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls podName:bc8aa2d9-ec75-4b25-9d02-e31a55b59338 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:07.402807358 +0000 UTC m=+39.752079729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls") pod "dns-default-mrj7p" (UID: "bc8aa2d9-ec75-4b25-9d02-e31a55b59338") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:06.086217 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:06.086177    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b96350d-9e01-4ec8-9ac7-3f4929a6eac4" containerID="3344a18bc6b82810241e9dfb9377ac42277f8977a6f18829781d6a1e63e5ae3c" exitCode=0
Sep 18 20:21:06.086767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:06.086235    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerDied","Data":"3344a18bc6b82810241e9dfb9377ac42277f8977a6f18829781d6a1e63e5ae3c"}
Sep 18 20:21:07.419559 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:07.419469    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:07.420028 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:07.419608    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:07.420028 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:07.419696    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls podName:bc8aa2d9-ec75-4b25-9d02-e31a55b59338 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:11.41967661 +0000 UTC m=+43.768948983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls") pod "dns-default-mrj7p" (UID: "bc8aa2d9-ec75-4b25-9d02-e31a55b59338") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:08.093060 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.092834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5" event={"ID":"c25929c2-99dc-4114-be12-f46793185392","Type":"ContainerStarted","Data":"724bfade01f4a2b4a9e4d622cfd744909ff8d6e9e3dbcdaf05f95e01f2178995"}
Sep 18 20:21:08.096099 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.096073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" event={"ID":"8b96350d-9e01-4ec8-9ac7-3f4929a6eac4","Type":"ContainerStarted","Data":"7859c3aabf5b4f4f6f6d5520f6d3321f8453088bb94b833f695379d0fe04a9e2"}
Sep 18 20:21:08.097543 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.097521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ddcx" event={"ID":"cb5ab505-4689-4946-bb53-1e5c4fffe7a9","Type":"ContainerStarted","Data":"69b3ca6cc5ce8eb67522eb8097f7ef5ef86de40b499ea77822e7fc74caeb20a3"}
Sep 18 20:21:08.111180 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.111144    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgmm5" podStartSLOduration=1.111121196 podStartE2EDuration="4.111132364s" totalImagesPullingTime="3.000011168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:08.109741099 +0000 UTC m=+40.459013493" watchObservedRunningTime="2026-09-18 20:21:08.111132364 +0000 UTC m=+40.460404771"
Sep 18 20:21:08.143430 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.143392    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7ddcx" podStartSLOduration=1.8613866479999999 podStartE2EDuration="5.143377826s" totalImagesPullingTime="3.281991178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:08.142343552 +0000 UTC m=+40.491615957" watchObservedRunningTime="2026-09-18 20:21:08.143377826 +0000 UTC m=+40.492650222"
Sep 18 20:21:08.173056 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.173008    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q9hpx" podStartSLOduration=9.90610556 podStartE2EDuration="38.17299683s" totalImagesPullingTime="28.26689127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:08.171587821 +0000 UTC m=+40.520860241" watchObservedRunningTime="2026-09-18 20:21:08.17299683 +0000 UTC m=+40.522269223"
Sep 18 20:21:08.717304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.717271    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vqldl"]
Sep 18 20:21:08.720338 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.720321    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.723149 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.723129    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 20:21:08.723696 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.723680    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 20:21:08.723777 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.723713    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 20:21:08.724472 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.724455    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-299f5\""
Sep 18 20:21:08.724553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.724458    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 20:21:08.734992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.734974    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqldl"]
Sep 18 20:21:08.821391 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.821357    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"]
Sep 18 20:21:08.824174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.824153    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"
Sep 18 20:21:08.827267 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827157    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-crio-socket\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.827267 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.827267 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827239    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 18 20:21:08.827545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827241    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 18 20:21:08.827545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-data-volume\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.827545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.827545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzsv\" (UniqueName: \"kubernetes.io/projected/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-api-access-mtzsv\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.827545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.827438    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c9smg\""
Sep 18 20:21:08.843092 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.843067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"]
Sep 18 20:21:08.928474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-crio-socket\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.928663 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928484    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnsbv\" (UniqueName: \"kubernetes.io/projected/c733f2b8-3449-45b5-91c3-5ea7db22ac77-kube-api-access-mnsbv\") pod \"migrator-76794cbd6c-8st7g\" (UID: \"c733f2b8-3449-45b5-91c3-5ea7db22ac77\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"
Sep 18 20:21:08.928663 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.928663 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:08.928625    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:08.928829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-data-volume\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.928829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-crio-socket\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.928829 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:08.928699    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.428680854 +0000 UTC m=+41.777953241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:08.928829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.928829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzsv\" (UniqueName: \"kubernetes.io/projected/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-api-access-mtzsv\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.929046 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:08.928863    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:21:08.929046 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:08.928930    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.428914892 +0000 UTC m=+41.778187279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : configmap "kube-rbac-proxy" not found
Sep 18 20:21:08.929046 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.928994    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-data-volume\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:08.941142 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:08.941117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzsv\" (UniqueName: \"kubernetes.io/projected/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-api-access-mtzsv\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:09.029888 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.029859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnsbv\" (UniqueName: \"kubernetes.io/projected/c733f2b8-3449-45b5-91c3-5ea7db22ac77-kube-api-access-mnsbv\") pod \"migrator-76794cbd6c-8st7g\" (UID: \"c733f2b8-3449-45b5-91c3-5ea7db22ac77\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"
Sep 18 20:21:09.035227 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.035206    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-w558x"]
Sep 18 20:21:09.038042 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.038028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.044835 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.044709    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 18 20:21:09.045222 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.045203    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 18 20:21:09.045308 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.045222    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2wdxw\""
Sep 18 20:21:09.046596 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.046578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnsbv\" (UniqueName: \"kubernetes.io/projected/c733f2b8-3449-45b5-91c3-5ea7db22ac77-kube-api-access-mnsbv\") pod \"migrator-76794cbd6c-8st7g\" (UID: \"c733f2b8-3449-45b5-91c3-5ea7db22ac77\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"
Sep 18 20:21:09.058671 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.058650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-w558x"]
Sep 18 20:21:09.130643 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.130597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.130827 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.130782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2c6f6f42-1937-4741-a7dd-5ba7b3710722-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.132644 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.132607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"
Sep 18 20:21:09.231260 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.231230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2c6f6f42-1937-4741-a7dd-5ba7b3710722-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.231413 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.231266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.231582 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.231550    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:21:09.231669 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.231651    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert podName:2c6f6f42-1937-4741-a7dd-5ba7b3710722 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.731608294 +0000 UTC m=+42.080880677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-w558x" (UID: "2c6f6f42-1937-4741-a7dd-5ba7b3710722") : secret "networking-console-plugin-cert" not found
Sep 18 20:21:09.231995 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.231975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2c6f6f42-1937-4741-a7dd-5ba7b3710722-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.406678 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.406589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g"]
Sep 18 20:21:09.410011 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:09.409979    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc733f2b8_3449_45b5_91c3_5ea7db22ac77.slice/crio-b4cb16e638c3084a97c6d3bb8c00a14a15bea1a9c3ed62f08b3a597ae9ee819f WatchSource:0}: Error finding container b4cb16e638c3084a97c6d3bb8c00a14a15bea1a9c3ed62f08b3a597ae9ee819f: Status 404 returned error can't find the container with id b4cb16e638c3084a97c6d3bb8c00a14a15bea1a9c3ed62f08b3a597ae9ee819f
Sep 18 20:21:09.432988 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.432966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:09.433077 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.433006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:09.433122 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.433114    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.433167 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.433157    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:10.433144422 +0000 UTC m=+42.782416793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.433214 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.433205    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.433270 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.433259    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:10.433246408 +0000 UTC m=+42.782518796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.734913 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:09.734832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:09.735298 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.734973    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:21:09.735298 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:09.735034    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert podName:2c6f6f42-1937-4741-a7dd-5ba7b3710722 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:10.735019648 +0000 UTC m=+43.084292035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-w558x" (UID: "2c6f6f42-1937-4741-a7dd-5ba7b3710722") : secret "networking-console-plugin-cert" not found
Sep 18 20:21:10.103795 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.103759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g" event={"ID":"c733f2b8-3449-45b5-91c3-5ea7db22ac77","Type":"ContainerStarted","Data":"b4cb16e638c3084a97c6d3bb8c00a14a15bea1a9c3ed62f08b3a597ae9ee819f"}
Sep 18 20:21:10.440418 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.440346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:10.440555 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.440432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:10.440555 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:10.440475    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:10.440555 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:10.440533    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:12.440516412 +0000 UTC m=+44.789788783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:10.440887 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.440869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:10.651207 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.651179    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8wqs_7e641a0c-60b0-40a9-bf65-7fea23ffdd29/dns-node-resolver/0.log"
Sep 18 20:21:10.741977 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:10.741905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:10.742301 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:10.742009    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:21:10.742301 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:10.742069    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert podName:2c6f6f42-1937-4741-a7dd-5ba7b3710722 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:12.742054996 +0000 UTC m=+45.091327367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-w558x" (UID: "2c6f6f42-1937-4741-a7dd-5ba7b3710722") : secret "networking-console-plugin-cert" not found
Sep 18 20:21:11.006492 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.006417    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vj8pv"]
Sep 18 20:21:11.009301 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.009286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.012226 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.012202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9v94m\""
Sep 18 20:21:11.012350 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.012209    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 20:21:11.012350 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.012210    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 20:21:11.013348 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.013330    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 20:21:11.013411 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.013355    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 20:21:11.017992 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.017970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vj8pv"]
Sep 18 20:21:11.108063 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.108036    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g" event={"ID":"c733f2b8-3449-45b5-91c3-5ea7db22ac77","Type":"ContainerStarted","Data":"6ead1f69bcb4695f70694644850cc8a7be70de3f53854e864a3bf5f9e38330f7"}
Sep 18 20:21:11.108191 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.108086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g" event={"ID":"c733f2b8-3449-45b5-91c3-5ea7db22ac77","Type":"ContainerStarted","Data":"25290c5bc0b05f07fe86659e57ef526153e41a2fb2e3284814e23fa9955405ae"}
Sep 18 20:21:11.145280 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.145251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-cabundle\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.145379 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.145289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sttqq\" (UniqueName: \"kubernetes.io/projected/e6889911-7ba4-43d1-b5d4-4e726be246de-kube-api-access-sttqq\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.145379 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.145345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-key\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.245948 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.245920    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-key\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.246038 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.246000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-cabundle\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.246072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.246035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sttqq\" (UniqueName: \"kubernetes.io/projected/e6889911-7ba4-43d1-b5d4-4e726be246de-kube-api-access-sttqq\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.246805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.246786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-cabundle\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.248321 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.248301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6889911-7ba4-43d1-b5d4-4e726be246de-signing-key\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.258996 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.258940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sttqq\" (UniqueName: \"kubernetes.io/projected/e6889911-7ba4-43d1-b5d4-4e726be246de-kube-api-access-sttqq\") pod \"service-ca-55f6fc7b86-vj8pv\" (UID: \"e6889911-7ba4-43d1-b5d4-4e726be246de\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.318491 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.318465    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv"
Sep 18 20:21:11.447193 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.447158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:11.447328 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:11.447263    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:11.447328 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:11.447326    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls podName:bc8aa2d9-ec75-4b25-9d02-e31a55b59338 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:19.447311239 +0000 UTC m=+51.796583610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls") pod "dns-default-mrj7p" (UID: "bc8aa2d9-ec75-4b25-9d02-e31a55b59338") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:11.459081 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.459055    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2hj9w_66791213-4af8-4a60-9399-19e12eb2b352/node-ca/0.log"
Sep 18 20:21:11.606761 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.606531    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-8st7g" podStartSLOduration=2.57267702 podStartE2EDuration="3.606513167s" totalImagesPullingTime="1.033836147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:11.131003134 +0000 UTC m=+43.480275556" watchObservedRunningTime="2026-09-18 20:21:11.606513167 +0000 UTC m=+43.955785559"
Sep 18 20:21:11.607344 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:11.607315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vj8pv"]
Sep 18 20:21:11.610491 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:11.610467    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6889911_7ba4_43d1_b5d4_4e726be246de.slice/crio-b12b2967b5f826fdcf374dc7ae3b5bd0acfcf160672c243b6b98cbc657c3ad83 WatchSource:0}: Error finding container b12b2967b5f826fdcf374dc7ae3b5bd0acfcf160672c243b6b98cbc657c3ad83: Status 404 returned error can't find the container with id b12b2967b5f826fdcf374dc7ae3b5bd0acfcf160672c243b6b98cbc657c3ad83
Sep 18 20:21:12.112673 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:12.112613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv" event={"ID":"e6889911-7ba4-43d1-b5d4-4e726be246de","Type":"ContainerStarted","Data":"b12b2967b5f826fdcf374dc7ae3b5bd0acfcf160672c243b6b98cbc657c3ad83"}
Sep 18 20:21:12.261754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:12.261718    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ddcx_cb5ab505-4689-4946-bb53-1e5c4fffe7a9/serve-healthcheck-canary/0.log"
Sep 18 20:21:12.456800 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:12.456705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:12.456964 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:12.456844    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:12.456964 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:12.456916    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:16.456896014 +0000 UTC m=+48.806168405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:12.758963 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:12.758871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:12.759141 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:12.759035    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:21:12.759141 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:12.759108    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert podName:2c6f6f42-1937-4741-a7dd-5ba7b3710722 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:16.759087961 +0000 UTC m=+49.108360351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-w558x" (UID: "2c6f6f42-1937-4741-a7dd-5ba7b3710722") : secret "networking-console-plugin-cert" not found
Sep 18 20:21:14.120515 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:14.120471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv" event={"ID":"e6889911-7ba4-43d1-b5d4-4e726be246de","Type":"ContainerStarted","Data":"bce02561c6b7cffdbd83b69ae8b460dd5dbb2d6c4bb68abe23bb63db58c6f4d6"}
Sep 18 20:21:16.488770 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:16.488733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:16.489166 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:16.488879    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:16.489166 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:16.488941    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls podName:9d8cb2d0-ec33-425c-a64c-8027c501a5ca nodeName:}" failed. No retries permitted until 2026-09-18 20:21:24.488925457 +0000 UTC m=+56.838197827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqldl" (UID: "9d8cb2d0-ec33-425c-a64c-8027c501a5ca") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:16.790593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:16.790556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:16.790802 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:16.790725    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:21:16.790802 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:16.790801    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert podName:2c6f6f42-1937-4741-a7dd-5ba7b3710722 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:24.790781472 +0000 UTC m=+57.140053858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-w558x" (UID: "2c6f6f42-1937-4741-a7dd-5ba7b3710722") : secret "networking-console-plugin-cert" not found
Sep 18 20:21:19.505944 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:19.505902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:19.508218 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:19.508197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc8aa2d9-ec75-4b25-9d02-e31a55b59338-metrics-tls\") pod \"dns-default-mrj7p\" (UID: \"bc8aa2d9-ec75-4b25-9d02-e31a55b59338\") " pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:19.603603 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:19.603566    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:19.923729 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:19.923681    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-vj8pv" podStartSLOduration=8.147764902 podStartE2EDuration="9.923664087s" totalImagesPullingTime="1.775899185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:14.136551546 +0000 UTC m=+46.485823956" watchObservedRunningTime="2026-09-18 20:21:19.923664087 +0000 UTC m=+52.272936479"
Sep 18 20:21:19.923927 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:19.923865    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mrj7p"]
Sep 18 20:21:19.927017 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:19.926988    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8aa2d9_ec75_4b25_9d02_e31a55b59338.slice/crio-930c98dd9ede916672904919c01d000ac295b2daa945c7cedd7b61d41bc34a39 WatchSource:0}: Error finding container 930c98dd9ede916672904919c01d000ac295b2daa945c7cedd7b61d41bc34a39: Status 404 returned error can't find the container with id 930c98dd9ede916672904919c01d000ac295b2daa945c7cedd7b61d41bc34a39
Sep 18 20:21:20.141512 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:20.141476    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrj7p" event={"ID":"bc8aa2d9-ec75-4b25-9d02-e31a55b59338","Type":"ContainerStarted","Data":"930c98dd9ede916672904919c01d000ac295b2daa945c7cedd7b61d41bc34a39"}
Sep 18 20:21:22.149828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:22.149745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrj7p" event={"ID":"bc8aa2d9-ec75-4b25-9d02-e31a55b59338","Type":"ContainerStarted","Data":"1805ab479833e3bbd1cd676af1e1decf745715e400d8905e13e74d91be8813b9"}
Sep 18 20:21:22.149828 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:22.149809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrj7p" event={"ID":"bc8aa2d9-ec75-4b25-9d02-e31a55b59338","Type":"ContainerStarted","Data":"405724688b3b95dc25ef505de0c8901c7ffefcf08ceca5b8c21002e9e72a17d8"}
Sep 18 20:21:22.150222 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:22.149837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:22.173703 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:22.173650    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mrj7p" podStartSLOduration=17.258344103 podStartE2EDuration="19.17361375s" totalImagesPullingTime="1.915269647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:22.172525304 +0000 UTC m=+54.521797918" watchObservedRunningTime="2026-09-18 20:21:22.17361375 +0000 UTC m=+54.522886143"
Sep 18 20:21:24.546080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.546041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:24.548381 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.548354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8cb2d0-ec33-425c-a64c-8027c501a5ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqldl\" (UID: \"9d8cb2d0-ec33-425c-a64c-8027c501a5ca\") " pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:24.630769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.630730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqldl"
Sep 18 20:21:24.849368 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.849280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:24.851657 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.851618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c6f6f42-1937-4741-a7dd-5ba7b3710722-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-w558x\" (UID: \"2c6f6f42-1937-4741-a7dd-5ba7b3710722\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:24.939677 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.939645    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqldl"]
Sep 18 20:21:24.947753 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:24.947731    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x"
Sep 18 20:21:24.955449 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:24.955422    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8cb2d0_ec33_425c_a64c_8027c501a5ca.slice/crio-59c37463e12b81b875669e3391affac988dfa81285a5ca2c48e3176b52e7c467 WatchSource:0}: Error finding container 59c37463e12b81b875669e3391affac988dfa81285a5ca2c48e3176b52e7c467: Status 404 returned error can't find the container with id 59c37463e12b81b875669e3391affac988dfa81285a5ca2c48e3176b52e7c467
Sep 18 20:21:25.161540 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:25.161457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqldl" event={"ID":"9d8cb2d0-ec33-425c-a64c-8027c501a5ca","Type":"ContainerStarted","Data":"5168d5c419f64a82ec245f07c13d16e9426291339ac5cce971ca7dd8173fbad0"}
Sep 18 20:21:25.161540 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:25.161519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqldl" event={"ID":"9d8cb2d0-ec33-425c-a64c-8027c501a5ca","Type":"ContainerStarted","Data":"59c37463e12b81b875669e3391affac988dfa81285a5ca2c48e3176b52e7c467"}
Sep 18 20:21:25.243963 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:25.243932    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-w558x"]
Sep 18 20:21:25.256523 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:25.256498    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6f6f42_1937_4741_a7dd_5ba7b3710722.slice/crio-71a6ac391f49160e7aec71e221603024ff3f87d8b181eb945cddf969c77b4b58 WatchSource:0}: Error finding container 71a6ac391f49160e7aec71e221603024ff3f87d8b181eb945cddf969c77b4b58: Status 404 returned error can't find the container with id 71a6ac391f49160e7aec71e221603024ff3f87d8b181eb945cddf969c77b4b58
Sep 18 20:21:26.169372 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:26.169332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x" event={"ID":"2c6f6f42-1937-4741-a7dd-5ba7b3710722","Type":"ContainerStarted","Data":"71a6ac391f49160e7aec71e221603024ff3f87d8b181eb945cddf969c77b4b58"}
Sep 18 20:21:27.175058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:27.174972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqldl" event={"ID":"9d8cb2d0-ec33-425c-a64c-8027c501a5ca","Type":"ContainerStarted","Data":"d72bfa1e4bae25ee67915b629a1620f99e2f88645261c4af937871831d333b00"}
Sep 18 20:21:27.176737 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:27.176706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x" event={"ID":"2c6f6f42-1937-4741-a7dd-5ba7b3710722","Type":"ContainerStarted","Data":"b179142932e3bd3fe42466c2e2faf38f4bb90e947ea01ea46403920c026cb360"}
Sep 18 20:21:27.199504 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:27.199455    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-w558x" podStartSLOduration=16.675399063 podStartE2EDuration="18.199433648s" totalImagesPullingTime="1.524034585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:27.198173942 +0000 UTC m=+59.547446355" watchObservedRunningTime="2026-09-18 20:21:27.199433648 +0000 UTC m=+59.548706040"
Sep 18 20:21:28.064107 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:28.064078    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nqtg"
Sep 18 20:21:29.184648 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:29.184601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqldl" event={"ID":"9d8cb2d0-ec33-425c-a64c-8027c501a5ca","Type":"ContainerStarted","Data":"01bee87126c143cfd1c42d50621fef5ceeb6ebed619a861a219a2db0cdf572a5"}
Sep 18 20:21:29.203562 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:29.203509    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vqldl" podStartSLOduration=18.157583633 podStartE2EDuration="21.203493832s" totalImagesPullingTime="3.045910199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:21:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:29.203110114 +0000 UTC m=+61.552382527" watchObservedRunningTime="2026-09-18 20:21:29.203493832 +0000 UTC m=+61.552766217"
Sep 18 20:21:32.154788 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:32.154751    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mrj7p"
Sep 18 20:21:34.728204 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.728166    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"]
Sep 18 20:21:34.731990 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.731966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.736930 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.736897    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 20:21:34.737064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.737040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 18 20:21:34.737064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.737050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 20:21:34.737193 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.737131    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bl5cb\""
Sep 18 20:21:34.737193 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.737050    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 20:21:34.745310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.745287    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"]
Sep 18 20:21:34.747467 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.747454    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.750178 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:34.750156    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"work-manager-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-140-180.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-140-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" type="*v1.Secret"
Sep 18 20:21:34.757558 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.757537    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"]
Sep 18 20:21:34.761218 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.761198    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"]
Sep 18 20:21:34.822694 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.822651    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9slp7\" (UniqueName: \"kubernetes.io/projected/e680d491-dd3b-4938-8720-a41a37fc7a3c-kube-api-access-9slp7\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.822694 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.822695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttr9v\" (UniqueName: \"kubernetes.io/projected/21de674d-597f-4ba0-9172-c81941333624-kube-api-access-ttr9v\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.822901 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.822774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21de674d-597f-4ba0-9172-c81941333624-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.822901 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.822802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e680d491-dd3b-4938-8720-a41a37fc7a3c-tmp\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.822901 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.822836    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e680d491-dd3b-4938-8720-a41a37fc7a3c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.924071 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttr9v\" (UniqueName: \"kubernetes.io/projected/21de674d-597f-4ba0-9172-c81941333624-kube-api-access-ttr9v\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.924248 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21de674d-597f-4ba0-9172-c81941333624-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.924248 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e680d491-dd3b-4938-8720-a41a37fc7a3c-tmp\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.924248 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e680d491-dd3b-4938-8720-a41a37fc7a3c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.924248 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9slp7\" (UniqueName: \"kubernetes.io/projected/e680d491-dd3b-4938-8720-a41a37fc7a3c-kube-api-access-9slp7\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.924673 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.924648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e680d491-dd3b-4938-8720-a41a37fc7a3c-tmp\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.926793 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.926771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21de674d-597f-4ba0-9172-c81941333624-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:34.948972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.948942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9slp7\" (UniqueName: \"kubernetes.io/projected/e680d491-dd3b-4938-8720-a41a37fc7a3c-kube-api-access-9slp7\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:34.949191 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:34.949002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttr9v\" (UniqueName: \"kubernetes.io/projected/21de674d-597f-4ba0-9172-c81941333624-kube-api-access-ttr9v\") pod \"managed-serviceaccount-addon-agent-59bff55bd-bgfrc\" (UID: \"21de674d-597f-4ba0-9172-c81941333624\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:35.049259 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.049228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"
Sep 18 20:21:35.319385 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.319306    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc"]
Sep 18 20:21:35.322269 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:35.322246    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21de674d_597f_4ba0_9172_c81941333624.slice/crio-5f53868e54c81d9d8007071522c4ef7c827efe97cc406bb3dab9863e6193e57e WatchSource:0}: Error finding container 5f53868e54c81d9d8007071522c4ef7c827efe97cc406bb3dab9863e6193e57e: Status 404 returned error can't find the container with id 5f53868e54c81d9d8007071522c4ef7c827efe97cc406bb3dab9863e6193e57e
Sep 18 20:21:35.529264 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.529217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:35.529470 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.529303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:35.532205 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.532185    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:21:35.532407 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.532390    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:21:35.542092 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.542056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17ece2c5-d02f-4898-871f-313bb14f89ed-metrics-certs\") pod \"network-metrics-daemon-s4rsp\" (UID: \"17ece2c5-d02f-4898-871f-313bb14f89ed\") " pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:35.542092 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.542083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/392944fa-d16b-411b-86a8-62c6985ee1a5-original-pull-secret\") pod \"global-pull-secret-syncer-7q8s6\" (UID: \"392944fa-d16b-411b-86a8-62c6985ee1a5\") " pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:35.629877 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.629791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:35.632230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.632212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zg9d\" (UniqueName: \"kubernetes.io/projected/8cabf748-848f-435c-af08-77e33aa4bd09-kube-api-access-7zg9d\") pod \"network-check-target-x5lgz\" (UID: \"8cabf748-848f-435c-af08-77e33aa4bd09\") " pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:35.688026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.687993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 20:21:35.697681 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.697658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e680d491-dd3b-4938-8720-a41a37fc7a3c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cb49cc97d-5rmxp\" (UID: \"e680d491-dd3b-4938-8720-a41a37fc7a3c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:35.702518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.702499    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q8s6"
Sep 18 20:21:35.704319 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.704294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27fhx\""
Sep 18 20:21:35.712121 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.712098    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4rsp"
Sep 18 20:21:35.901981 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.901903    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmjzr\""
Sep 18 20:21:35.908978 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.908959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:35.956150 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.955890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:35.987653 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:35.987617    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q8s6"]
Sep 18 20:21:35.990093 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:35.990069    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392944fa_d16b_411b_86a8_62c6985ee1a5.slice/crio-cc7c700e271d06e5880358efec0450313af9bde3bf7c1c1245247cdb8a9d61da WatchSource:0}: Error finding container cc7c700e271d06e5880358efec0450313af9bde3bf7c1c1245247cdb8a9d61da: Status 404 returned error can't find the container with id cc7c700e271d06e5880358efec0450313af9bde3bf7c1c1245247cdb8a9d61da
Sep 18 20:21:36.003829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.003729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s4rsp"]
Sep 18 20:21:36.008316 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:36.008274    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ece2c5_d02f_4898_871f_313bb14f89ed.slice/crio-6e6b7ec948653556e7179a66ec457ef222ca53c03f472d72c5f90b9268039d33 WatchSource:0}: Error finding container 6e6b7ec948653556e7179a66ec457ef222ca53c03f472d72c5f90b9268039d33: Status 404 returned error can't find the container with id 6e6b7ec948653556e7179a66ec457ef222ca53c03f472d72c5f90b9268039d33
Sep 18 20:21:36.178132 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.178006    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x5lgz"]
Sep 18 20:21:36.180246 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:36.180219    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cabf748_848f_435c_af08_77e33aa4bd09.slice/crio-4fb071b6d3071f92bfe386220a251ce706cfdaeaa986473be1ed1aa98cf2a858 WatchSource:0}: Error finding container 4fb071b6d3071f92bfe386220a251ce706cfdaeaa986473be1ed1aa98cf2a858: Status 404 returned error can't find the container with id 4fb071b6d3071f92bfe386220a251ce706cfdaeaa986473be1ed1aa98cf2a858
Sep 18 20:21:36.207271 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.207245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4rsp" event={"ID":"17ece2c5-d02f-4898-871f-313bb14f89ed","Type":"ContainerStarted","Data":"6e6b7ec948653556e7179a66ec457ef222ca53c03f472d72c5f90b9268039d33"}
Sep 18 20:21:36.208303 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.208277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc" event={"ID":"21de674d-597f-4ba0-9172-c81941333624","Type":"ContainerStarted","Data":"5f53868e54c81d9d8007071522c4ef7c827efe97cc406bb3dab9863e6193e57e"}
Sep 18 20:21:36.209307 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.209287    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x5lgz" event={"ID":"8cabf748-848f-435c-af08-77e33aa4bd09","Type":"ContainerStarted","Data":"4fb071b6d3071f92bfe386220a251ce706cfdaeaa986473be1ed1aa98cf2a858"}
Sep 18 20:21:36.210327 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.210309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q8s6" event={"ID":"392944fa-d16b-411b-86a8-62c6985ee1a5","Type":"ContainerStarted","Data":"cc7c700e271d06e5880358efec0450313af9bde3bf7c1c1245247cdb8a9d61da"}
Sep 18 20:21:36.229748 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:36.229727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"]
Sep 18 20:21:36.232901 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:36.232878    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode680d491_dd3b_4938_8720_a41a37fc7a3c.slice/crio-9132595c65ad48f1e269a1c368690715bdcc23464a13b15d7842dd2e6e21f542 WatchSource:0}: Error finding container 9132595c65ad48f1e269a1c368690715bdcc23464a13b15d7842dd2e6e21f542: Status 404 returned error can't find the container with id 9132595c65ad48f1e269a1c368690715bdcc23464a13b15d7842dd2e6e21f542
Sep 18 20:21:37.217895 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:37.217858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x5lgz" event={"ID":"8cabf748-848f-435c-af08-77e33aa4bd09","Type":"ContainerStarted","Data":"49cc477caab137a59cbc8e71b24661d4cca3bbe013444c67c5a4b4648ea7ae2a"}
Sep 18 20:21:37.218286 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:37.218187    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:21:37.220655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:37.220610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp" event={"ID":"e680d491-dd3b-4938-8720-a41a37fc7a3c","Type":"ContainerStarted","Data":"9132595c65ad48f1e269a1c368690715bdcc23464a13b15d7842dd2e6e21f542"}
Sep 18 20:21:37.251998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:37.251949    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x5lgz" podStartSLOduration=67.25191122 podStartE2EDuration="1m7.25191122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:37.251201669 +0000 UTC m=+69.600474061" watchObservedRunningTime="2026-09-18 20:21:37.25191122 +0000 UTC m=+69.601183612"
Sep 18 20:21:38.228123 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.227668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4rsp" event={"ID":"17ece2c5-d02f-4898-871f-313bb14f89ed","Type":"ContainerStarted","Data":"b80ad8b7e57242e0c52a35987be8c52a541e72e54b6940ff5c514ea273879148"}
Sep 18 20:21:38.228123 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.227726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4rsp" event={"ID":"17ece2c5-d02f-4898-871f-313bb14f89ed","Type":"ContainerStarted","Data":"f3cbc6be49b95444647c3f0162c025412b7fae4c041cc515421bd4158d63f691"}
Sep 18 20:21:38.645884 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.645788    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s4rsp" podStartSLOduration=67.527249042 podStartE2EDuration="1m8.645763241s" totalImagesPullingTime="1.118514199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:38.256477415 +0000 UTC m=+70.605749835" watchObservedRunningTime="2026-09-18 20:21:38.645763241 +0000 UTC m=+70.995035648"
Sep 18 20:21:38.646649 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.646602    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"]
Sep 18 20:21:38.649946 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.649925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:38.653493 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.653215    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 18 20:21:38.653493 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.653222    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n4v8k\""
Sep 18 20:21:38.675301 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.675270    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"]
Sep 18 20:21:38.757490 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.757457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/895bd099-6c92-498e-95f1-bea90c3b907e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-4p859\" (UID: \"895bd099-6c92-498e-95f1-bea90c3b907e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:38.858932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.858889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/895bd099-6c92-498e-95f1-bea90c3b907e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-4p859\" (UID: \"895bd099-6c92-498e-95f1-bea90c3b907e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:38.861840 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.861814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/895bd099-6c92-498e-95f1-bea90c3b907e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-4p859\" (UID: \"895bd099-6c92-498e-95f1-bea90c3b907e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:38.963505 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:38.963430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:40.596356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:40.596139    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"]
Sep 18 20:21:40.600567 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:40.600534    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895bd099_6c92_498e_95f1_bea90c3b907e.slice/crio-2d2a658a5c994ab4842ff81f28ac35f1326db36b4f7c96173d756434cac22943 WatchSource:0}: Error finding container 2d2a658a5c994ab4842ff81f28ac35f1326db36b4f7c96173d756434cac22943: Status 404 returned error can't find the container with id 2d2a658a5c994ab4842ff81f28ac35f1326db36b4f7c96173d756434cac22943
Sep 18 20:21:41.243304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:41.243267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q8s6" event={"ID":"392944fa-d16b-411b-86a8-62c6985ee1a5","Type":"ContainerStarted","Data":"968950f0b209545e3740a165568d148c796066dfc2b84021a88837bcbbdea28e"}
Sep 18 20:21:41.244860 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:41.244832    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859" event={"ID":"895bd099-6c92-498e-95f1-bea90c3b907e","Type":"ContainerStarted","Data":"2d2a658a5c994ab4842ff81f28ac35f1326db36b4f7c96173d756434cac22943"}
Sep 18 20:21:41.246594 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:41.246564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc" event={"ID":"21de674d-597f-4ba0-9172-c81941333624","Type":"ContainerStarted","Data":"4954b8053af222724f540d5c4cef8f01090bf3093b7b4992805732ea59b25710"}
Sep 18 20:21:41.293541 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:41.293489    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7q8s6" podStartSLOduration=66.949590078 podStartE2EDuration="1m11.29347017s" totalImagesPullingTime="4.343880092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:41.272337782 +0000 UTC m=+73.621610196" watchObservedRunningTime="2026-09-18 20:21:41.29347017 +0000 UTC m=+73.642742563"
Sep 18 20:21:41.294607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:41.294572    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59bff55bd-bgfrc" podStartSLOduration=2.300325477 podStartE2EDuration="7.294562068s" totalImagesPullingTime="4.994236591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:41.292596468 +0000 UTC m=+73.641868862" watchObservedRunningTime="2026-09-18 20:21:41.294562068 +0000 UTC m=+73.643834459"
Sep 18 20:21:43.255544 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.255508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859" event={"ID":"895bd099-6c92-498e-95f1-bea90c3b907e","Type":"ContainerStarted","Data":"d77c7f2a800957f0ce2ff246e305b6fe8c344376a3f9c562d6eda84dbd635622"}
Sep 18 20:21:43.256015 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.255669    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:43.257308 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.257279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp" event={"ID":"e680d491-dd3b-4938-8720-a41a37fc7a3c","Type":"ContainerStarted","Data":"7e440992e23c9eb1a9db85ce7d4e05a4bf6fd200bb53eac35307210223d86b64"}
Sep 18 20:21:43.257439 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.257398    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:43.258760 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.258740    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp"
Sep 18 20:21:43.260858 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.260840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859"
Sep 18 20:21:43.277946 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.277901    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-4p859" podStartSLOduration=2.8430308159999997 podStartE2EDuration="5.27788895s" totalImagesPullingTime="2.434858134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:43.2774927 +0000 UTC m=+75.626765094" watchObservedRunningTime="2026-09-18 20:21:43.27788895 +0000 UTC m=+75.627161335"
Sep 18 20:21:43.307536 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.307459    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cb49cc97d-5rmxp" podStartSLOduration=2.494363679 podStartE2EDuration="9.307444676s" totalImagesPullingTime="6.813080997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:43.306377277 +0000 UTC m=+75.655649683" watchObservedRunningTime="2026-09-18 20:21:43.307444676 +0000 UTC m=+75.656717069"
Sep 18 20:21:43.746822 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.746738    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"]
Sep 18 20:21:43.750292 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.750272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.754164 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.754137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 20:21:43.754293 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.754137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lp4r2\""
Sep 18 20:21:43.755395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.755366    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 20:21:43.755395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.755374    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 20:21:43.755395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.755398    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 20:21:43.757173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.757155    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 20:21:43.764233 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.764209    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"]
Sep 18 20:21:43.800623 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.800593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rqg\" (UniqueName: \"kubernetes.io/projected/71287f8c-e670-40a3-9cc9-ea3e21c09062-kube-api-access-f8rqg\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.800769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.800647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.800769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.800694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.800769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.800714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71287f8c-e670-40a3-9cc9-ea3e21c09062-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.901887 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.901848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.902082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.901902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71287f8c-e670-40a3-9cc9-ea3e21c09062-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.902082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.901931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rqg\" (UniqueName: \"kubernetes.io/projected/71287f8c-e670-40a3-9cc9-ea3e21c09062-kube-api-access-f8rqg\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.902082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.901973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.902651 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.902599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71287f8c-e670-40a3-9cc9-ea3e21c09062-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.904297 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.904273    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.904451 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.904428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/71287f8c-e670-40a3-9cc9-ea3e21c09062-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:43.912402 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:43.912371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rqg\" (UniqueName: \"kubernetes.io/projected/71287f8c-e670-40a3-9cc9-ea3e21c09062-kube-api-access-f8rqg\") pod \"prometheus-operator-7bd5d56d6-42h26\" (UID: \"71287f8c-e670-40a3-9cc9-ea3e21c09062\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:44.058963 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:44.058935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"
Sep 18 20:21:44.329539 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:44.329466    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-42h26"]
Sep 18 20:21:44.332228 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:44.332206    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71287f8c_e670_40a3_9cc9_ea3e21c09062.slice/crio-2fc1ee63a5621843275a9e9bacff59dd643c3c9e2459e98b6d25c6f83c0a030e WatchSource:0}: Error finding container 2fc1ee63a5621843275a9e9bacff59dd643c3c9e2459e98b6d25c6f83c0a030e: Status 404 returned error can't find the container with id 2fc1ee63a5621843275a9e9bacff59dd643c3c9e2459e98b6d25c6f83c0a030e
Sep 18 20:21:45.267128 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.267093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26" event={"ID":"71287f8c-e670-40a3-9cc9-ea3e21c09062","Type":"ContainerStarted","Data":"2fc1ee63a5621843275a9e9bacff59dd643c3c9e2459e98b6d25c6f83c0a030e"}
Sep 18 20:21:45.351175 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.351145    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:21:45.354526 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.354498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.358030 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.358009    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 20:21:45.358166 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.358044    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:21:45.359329 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.359305    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:21:45.359425 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.359410    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 20:21:45.359567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.359544    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 20:21:45.359668 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.359578    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 20:21:45.360798 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.360777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 20:21:45.360894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.360876    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-88xm9\""
Sep 18 20:21:45.364574 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.364559    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 20:21:45.367374 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.367351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:21:45.414238 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcwbr\" (UniqueName: \"kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414349    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.414553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.414401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515063 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.514999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515063 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515063 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515234 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515234 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcwbr\" (UniqueName: \"kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515234 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.515234 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.515177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.516117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.516056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.516225 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.516134    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.516323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.516300    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.516393 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.516363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.517812 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.517794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.518082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.518062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.525228 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.525204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcwbr\" (UniqueName: \"kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr\") pod \"console-976b66477-phssh\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") " pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.666147 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.666106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-976b66477-phssh"
Sep 18 20:21:45.947907 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:45.947875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:21:45.950834 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:45.950806    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3f5d8a_c5ad_488a_8ea9_41ecb2dabbb9.slice/crio-55f7694821f8c6081cfe230c62462e45987f3b97269c1333a6c6ea0ed461967a WatchSource:0}: Error finding container 55f7694821f8c6081cfe230c62462e45987f3b97269c1333a6c6ea0ed461967a: Status 404 returned error can't find the container with id 55f7694821f8c6081cfe230c62462e45987f3b97269c1333a6c6ea0ed461967a
Sep 18 20:21:46.276834 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:46.276743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26" event={"ID":"71287f8c-e670-40a3-9cc9-ea3e21c09062","Type":"ContainerStarted","Data":"3f1d84fcf705233fe834d2296e937b75d80e073d8a67f061c2ceeeb051b76f0d"}
Sep 18 20:21:46.276834 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:46.276831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26" event={"ID":"71287f8c-e670-40a3-9cc9-ea3e21c09062","Type":"ContainerStarted","Data":"09a506506ab53ff88935285ce5beb264e3bf18419be132a21c9a5178e2e6d6d4"}
Sep 18 20:21:46.277925 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:46.277899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-976b66477-phssh" event={"ID":"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9","Type":"ContainerStarted","Data":"55f7694821f8c6081cfe230c62462e45987f3b97269c1333a6c6ea0ed461967a"}
Sep 18 20:21:46.314015 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:46.313965    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-42h26" podStartSLOduration=2.212552452 podStartE2EDuration="3.313951509s" totalImagesPullingTime="1.101399057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:46.313792905 +0000 UTC m=+78.663065330" watchObservedRunningTime="2026-09-18 20:21:46.313951509 +0000 UTC m=+78.663223902"
Sep 18 20:21:48.205509 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.205475    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"]
Sep 18 20:21:48.212174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.212152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.218094 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.218071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 20:21:48.218254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.218071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 20:21:48.239736 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.239711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hcfz6\""
Sep 18 20:21:48.251384 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.251315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"]
Sep 18 20:21:48.315379 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.315350    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kmf8x"]
Sep 18 20:21:48.320576 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.320555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.329107 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.328996    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 20:21:48.329246 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.329220    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 20:21:48.329394 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.329369    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 20:21:48.329507 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.329492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-42k9n\""
Sep 18 20:21:48.338904 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.338881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5vjz\" (UniqueName: \"kubernetes.io/projected/66f9638b-fc9a-495f-a586-c16ae1200516-kube-api-access-k5vjz\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.339025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.338927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.339025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.338977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.339025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.339004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f9638b-fc9a-495f-a586-c16ae1200516-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.439991 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.439957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-wtmp\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.440189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f9638b-fc9a-495f-a586-c16ae1200516-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.440189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-textfile\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-sys\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440450 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:48.440170    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 18 20:21:48.440450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-metrics-client-ca\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5vjz\" (UniqueName: \"kubernetes.io/projected/66f9638b-fc9a-495f-a586-c16ae1200516-kube-api-access-k5vjz\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.440450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-root\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440450 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:48.440265    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls podName:66f9638b-fc9a-495f-a586-c16ae1200516 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:48.940244363 +0000 UTC m=+81.289516751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-pwsvk" (UID: "66f9638b-fc9a-495f-a586-c16ae1200516") : secret "openshift-state-metrics-tls" not found
Sep 18 20:21:48.440729 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440619    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440729 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.440827 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.440827 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f9638b-fc9a-495f-a586-c16ae1200516-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.440930 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.440836    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs8sl\" (UniqueName: \"kubernetes.io/projected/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-kube-api-access-bs8sl\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.443258 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.443236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.451859 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.451818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5vjz\" (UniqueName: \"kubernetes.io/projected/66f9638b-fc9a-495f-a586-c16ae1200516-kube-api-access-k5vjz\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.542072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.541943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-wtmp\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-textfile\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-sys\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-wtmp\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-metrics-client-ca\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-root\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542356 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542598 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542598 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-root\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542598 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs8sl\" (UniqueName: \"kubernetes.io/projected/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-kube-api-access-bs8sl\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-textfile\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-metrics-client-ca\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-sys\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.542778 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:48.542769    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 20:21:48.542970 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:21:48.542831    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls podName:8f6a09a2-1ec0-41fb-8761-229f9f7e9be4 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:49.042813378 +0000 UTC m=+81.392085757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls") pod "node-exporter-kmf8x" (UID: "8f6a09a2-1ec0-41fb-8761-229f9f7e9be4") : secret "node-exporter-tls" not found
Sep 18 20:21:48.542970 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.542850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.545464 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.545442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.560137 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.560112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs8sl\" (UniqueName: \"kubernetes.io/projected/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-kube-api-access-bs8sl\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:48.945760 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.945649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:48.948451 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:48.948412    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f9638b-fc9a-495f-a586-c16ae1200516-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pwsvk\" (UID: \"66f9638b-fc9a-495f-a586-c16ae1200516\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:49.046544 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.046496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:49.049178 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.049142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f6a09a2-1ec0-41fb-8761-229f9f7e9be4-node-exporter-tls\") pod \"node-exporter-kmf8x\" (UID: \"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4\") " pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:49.124187 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.124158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"
Sep 18 20:21:49.239037 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.238957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kmf8x"
Sep 18 20:21:49.612069 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.612033    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:21:49.619831 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.619797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.624528 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.624473    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 20:21:49.624528 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.624526    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 20:21:49.624758 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.624539    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 20:21:49.624758 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.624479    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 20:21:49.624868 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.624838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p7pcc\""
Sep 18 20:21:49.625478 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.625284    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 20:21:49.625478 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.625354    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 20:21:49.625478 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.625387    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 20:21:49.625699 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.625506    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 20:21:49.625699 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.625651    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 20:21:49.644858 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.644829    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:21:49.752607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-config-volume\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7cc\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-kube-api-access-hj7cc\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-web-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.752951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.753185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.752968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.753185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.753045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.753185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.753089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.753185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.753128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-config-out\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853709 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-config-out\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.853890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.853860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-config-volume\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7cc\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-kube-api-access-hj7cc\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-web-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.854951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.855651    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856772 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.856755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.856940 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.856834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f07853bf-849d-447c-9acb-f98e2c4f2121-config-out\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.857003 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.856932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.857697 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.857656    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.858199 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.858180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-config-volume\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.858294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.858274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.858998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.858973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.859417 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.859400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.859545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.859526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f07853bf-849d-447c-9acb-f98e2c4f2121-web-config\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.872781 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.872680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7cc\" (UniqueName: \"kubernetes.io/projected/f07853bf-849d-447c-9acb-f98e2c4f2121-kube-api-access-hj7cc\") pod \"alertmanager-main-0\" (UID: \"f07853bf-849d-447c-9acb-f98e2c4f2121\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.930787 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:49.930761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:21:49.993198 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:49.993169    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6a09a2_1ec0_41fb_8761_229f9f7e9be4.slice/crio-08fa4c332ca6eaa0151469dfc43aa3173b9643d1d6299cf6ef0aba49d12213ca WatchSource:0}: Error finding container 08fa4c332ca6eaa0151469dfc43aa3173b9643d1d6299cf6ef0aba49d12213ca: Status 404 returned error can't find the container with id 08fa4c332ca6eaa0151469dfc43aa3173b9643d1d6299cf6ef0aba49d12213ca
Sep 18 20:21:50.271433 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.271020    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk"]
Sep 18 20:21:50.273899 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:50.273861    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f9638b_fc9a_495f_a586_c16ae1200516.slice/crio-00c8c594664c23d0b98859a4fc2e2d325367e39fea6fcceb8ca26450c550d61d WatchSource:0}: Error finding container 00c8c594664c23d0b98859a4fc2e2d325367e39fea6fcceb8ca26450c550d61d: Status 404 returned error can't find the container with id 00c8c594664c23d0b98859a4fc2e2d325367e39fea6fcceb8ca26450c550d61d
Sep 18 20:21:50.282173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.282152    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:21:50.283006 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:50.282973    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07853bf_849d_447c_9acb_f98e2c4f2121.slice/crio-ec3b301c61d941a8e6a2f525d560fcd52872fe78dfbe85b47039918ff92bf3cf WatchSource:0}: Error finding container ec3b301c61d941a8e6a2f525d560fcd52872fe78dfbe85b47039918ff92bf3cf: Status 404 returned error can't find the container with id ec3b301c61d941a8e6a2f525d560fcd52872fe78dfbe85b47039918ff92bf3cf
Sep 18 20:21:50.297950 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.297923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk" event={"ID":"66f9638b-fc9a-495f-a586-c16ae1200516","Type":"ContainerStarted","Data":"00c8c594664c23d0b98859a4fc2e2d325367e39fea6fcceb8ca26450c550d61d"}
Sep 18 20:21:50.299116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.299094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"ec3b301c61d941a8e6a2f525d560fcd52872fe78dfbe85b47039918ff92bf3cf"}
Sep 18 20:21:50.300520 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.300495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-976b66477-phssh" event={"ID":"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9","Type":"ContainerStarted","Data":"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"}
Sep 18 20:21:50.301518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.301500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kmf8x" event={"ID":"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4","Type":"ContainerStarted","Data":"08fa4c332ca6eaa0151469dfc43aa3173b9643d1d6299cf6ef0aba49d12213ca"}
Sep 18 20:21:50.333024 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:50.332978    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-976b66477-phssh" podStartSLOduration=1.229942832 podStartE2EDuration="5.332963275s" totalImagesPullingTime="4.103020443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:50.332182454 +0000 UTC m=+82.681454846" watchObservedRunningTime="2026-09-18 20:21:50.332963275 +0000 UTC m=+82.682235670"
Sep 18 20:21:51.312461 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:51.311778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kmf8x" event={"ID":"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4","Type":"ContainerStarted","Data":"9e694f00c0472812f4819088b923961ae3da37710510ae64be8a2b9d29591b7d"}
Sep 18 20:21:51.315472 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:51.315444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk" event={"ID":"66f9638b-fc9a-495f-a586-c16ae1200516","Type":"ContainerStarted","Data":"7732f69964fe90fe9d74103770d74f34cd5ce9729ed6228c62f9423ff676abe1"}
Sep 18 20:21:51.316697 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:51.316675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk" event={"ID":"66f9638b-fc9a-495f-a586-c16ae1200516","Type":"ContainerStarted","Data":"4d1f7cb2613c83526913aeb91f6e91013e50133496b1de38091f44148e4dd7ba"}
Sep 18 20:21:52.320738 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.320702    2564 generic.go:356] "Generic (PLEG): container finished" podID="8f6a09a2-1ec0-41fb-8761-229f9f7e9be4" containerID="9e694f00c0472812f4819088b923961ae3da37710510ae64be8a2b9d29591b7d" exitCode=0
Sep 18 20:21:52.321162 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.320783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kmf8x" event={"ID":"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4","Type":"ContainerDied","Data":"9e694f00c0472812f4819088b923961ae3da37710510ae64be8a2b9d29591b7d"}
Sep 18 20:21:52.323055 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.323025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk" event={"ID":"66f9638b-fc9a-495f-a586-c16ae1200516","Type":"ContainerStarted","Data":"9107a0806174f15c0a5928f47e550495e6adc9348c499235c4fe67facc49e191"}
Sep 18 20:21:52.324609 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.324585    2564 generic.go:356] "Generic (PLEG): container finished" podID="f07853bf-849d-447c-9acb-f98e2c4f2121" containerID="3625fdfc1a4c286f31000b778ba52caf4c437a43f6bc13005ef79a20a9d27860" exitCode=0
Sep 18 20:21:52.324718 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.324648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerDied","Data":"3625fdfc1a4c286f31000b778ba52caf4c437a43f6bc13005ef79a20a9d27860"}
Sep 18 20:21:52.367671 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:52.367455    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pwsvk" podStartSLOduration=3.256174171 podStartE2EDuration="4.367433748s" totalImagesPullingTime="1.111259577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:52.366424394 +0000 UTC m=+84.715696812" watchObservedRunningTime="2026-09-18 20:21:52.367433748 +0000 UTC m=+84.716706143"
Sep 18 20:21:53.022355 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.022320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d8f95559c-qslpk"]
Sep 18 20:21:53.026154 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.026132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.029494 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.029460    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 20:21:53.029619 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.029520    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-db4n2\""
Sep 18 20:21:53.030808 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.030721    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 20:21:53.030912 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.030822    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 20:21:53.030912 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.030829    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 20:21:53.031111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.031092    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-29cem5asl48ak\""
Sep 18 20:21:53.039553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.039532    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8f95559c-qslpk"]
Sep 18 20:21:53.183712 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-metrics-server-audit-profiles\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.183858 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-client-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.183858 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a08a2763-7271-48c5-a7e6-4e8170c3c45b-audit-log\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.183858 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78r5q\" (UniqueName: \"kubernetes.io/projected/a08a2763-7271-48c5-a7e6-4e8170c3c45b-kube-api-access-78r5q\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.183978 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.183978 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-tls\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.184043 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.183983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-client-certs\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.284879 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.284843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a08a2763-7271-48c5-a7e6-4e8170c3c45b-audit-log\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285042 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.284919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78r5q\" (UniqueName: \"kubernetes.io/projected/a08a2763-7271-48c5-a7e6-4e8170c3c45b-kube-api-access-78r5q\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285042 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.284946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285042 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.285035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-tls\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285154 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.285063    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-client-certs\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285154 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.285100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-metrics-server-audit-profiles\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285154 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.285139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-client-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.285283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.285263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a08a2763-7271-48c5-a7e6-4e8170c3c45b-audit-log\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.286254 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.286224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.286481 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.286453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a08a2763-7271-48c5-a7e6-4e8170c3c45b-metrics-server-audit-profiles\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.288089 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.288045    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-client-certs\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.288599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.288563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-client-ca-bundle\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.288599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.288569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a08a2763-7271-48c5-a7e6-4e8170c3c45b-secret-metrics-server-tls\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.298170 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.298144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78r5q\" (UniqueName: \"kubernetes.io/projected/a08a2763-7271-48c5-a7e6-4e8170c3c45b-kube-api-access-78r5q\") pod \"metrics-server-5d8f95559c-qslpk\" (UID: \"a08a2763-7271-48c5-a7e6-4e8170c3c45b\") " pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.331590 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.331515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kmf8x" event={"ID":"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4","Type":"ContainerStarted","Data":"b4c14d656772c707e598122060678f15d7a1886bb5a7bb73271ea508550ca163"}
Sep 18 20:21:53.331999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.331615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kmf8x" event={"ID":"8f6a09a2-1ec0-41fb-8761-229f9f7e9be4","Type":"ContainerStarted","Data":"de2f2960bbd9b50f2aed975e2c1ddf6b0fa51870b658ebc56eb133189e5836be"}
Sep 18 20:21:53.337018 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.336997    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:21:53.360406 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.360366    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kmf8x" podStartSLOduration=4.104155394 podStartE2EDuration="5.360352499s" totalImagesPullingTime="1.182294157s" totalInitContainerRuntime="73.902948ms" isStatefulPod=true podCreationTimestamp="2026-09-18 20:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:53.358421677 +0000 UTC m=+85.707694094" watchObservedRunningTime="2026-09-18 20:21:53.360352499 +0000 UTC m=+85.709624936"
Sep 18 20:21:53.813266 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:53.813241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8f95559c-qslpk"]
Sep 18 20:21:53.819609 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:21:53.819584    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08a2763_7271_48c5_a7e6_4e8170c3c45b.slice/crio-520efb0f502c251ef04d7c695facbaae055af4fd7c4fba038ae447d96aaf1448 WatchSource:0}: Error finding container 520efb0f502c251ef04d7c695facbaae055af4fd7c4fba038ae447d96aaf1448: Status 404 returned error can't find the container with id 520efb0f502c251ef04d7c695facbaae055af4fd7c4fba038ae447d96aaf1448
Sep 18 20:21:54.342196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.341890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"7fadb933a2692e010b413dbaedac1df5444ce518c489c71426cc01e33ee910b4"}
Sep 18 20:21:54.342196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.341975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"8aba532932ca87f9ce7edbd2365f670a8db601e7eaa5cf062ccf15cfeded464b"}
Sep 18 20:21:54.342196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.342013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"0f50e412e8c72ee0f353b27c1682c7b9f66696b7a04f04b274ea56b0363a1de1"}
Sep 18 20:21:54.342196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.342048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"3144cec143ba4f47f89e60bf6dc7655f924a4f40752edf371d19725681ebec6e"}
Sep 18 20:21:54.342196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.342085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"0ff799d2831d422e0ad495ad036aeb7e54e9d706aa32d6cdf7b95267046a0a4f"}
Sep 18 20:21:54.344775 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.344750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk" event={"ID":"a08a2763-7271-48c5-a7e6-4e8170c3c45b","Type":"ContainerStarted","Data":"520efb0f502c251ef04d7c695facbaae055af4fd7c4fba038ae447d96aaf1448"}
Sep 18 20:21:54.564078 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:54.564042    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:21:55.357545 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:55.357508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f07853bf-849d-447c-9acb-f98e2c4f2121","Type":"ContainerStarted","Data":"ce36158d61b6a249d166d8331eb600abeaf00034af37a52c51ad8d227915f758"}
Sep 18 20:21:55.359283 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:55.359254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk" event={"ID":"a08a2763-7271-48c5-a7e6-4e8170c3c45b","Type":"ContainerStarted","Data":"72d2575a96c20321d4becfa8dbe0e5e75f532652b1a80192d9f1303094b57395"}
Sep 18 20:21:55.393292 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:55.393241    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.599638535 podStartE2EDuration="6.393222068s" totalImagesPullingTime="3.775605092s" totalInitContainerRuntime="17.978441ms" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:55.390116043 +0000 UTC m=+87.739388460" watchObservedRunningTime="2026-09-18 20:21:55.393222068 +0000 UTC m=+87.742494460"
Sep 18 20:21:55.420624 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:55.420259    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk" podStartSLOduration=1.982332464 podStartE2EDuration="3.420231227s" totalImagesPullingTime="1.437898763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:55.420187125 +0000 UTC m=+87.769459519" watchObservedRunningTime="2026-09-18 20:21:55.420231227 +0000 UTC m=+87.769503611"
Sep 18 20:21:55.667003 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:21:55.666972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-976b66477-phssh"
Sep 18 20:22:08.230452 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:08.230421    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x5lgz"
Sep 18 20:22:13.338081 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:13.338034    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:22:13.338538 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:13.338138    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:22:19.585717 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.585643    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-976b66477-phssh" podUID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" containerName="console" containerID="cri-o://8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf" gracePeriod=15
Sep 18 20:22:19.834770 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.834747    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-976b66477-phssh"
Sep 18 20:22:19.900192 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900124    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900192 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900241    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900269    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcwbr\" (UniqueName: \"kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900303    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900335    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config\") pod \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\" (UID: \"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9\") "
Sep 18 20:22:19.900590 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config" (OuterVolumeSpecName: "console-config") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:19.900772 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900742    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca" (OuterVolumeSpecName: "service-ca") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:19.900878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900779    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:19.900878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.900811    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:19.902505 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.902480    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:19.902689 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.902665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr" (OuterVolumeSpecName: "kube-api-access-vcwbr") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "kube-api-access-vcwbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:19.902843 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:19.902813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" (UID: "3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:20.001284 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001255    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-service-ca\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001284 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001278    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001284 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001288    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-trusted-ca-bundle\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001476 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001296    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcwbr\" (UniqueName: \"kubernetes.io/projected/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-kube-api-access-vcwbr\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001476 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001305    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-oauth-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001476 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001318    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-oauth-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.001476 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.001329    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9-console-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.456225 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.456197    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" containerID="8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf" exitCode=0
Sep 18 20:22:20.456371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.456234    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-976b66477-phssh"
Sep 18 20:22:20.456371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.456282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-976b66477-phssh" event={"ID":"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9","Type":"ContainerDied","Data":"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"}
Sep 18 20:22:20.456371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.456329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-976b66477-phssh" event={"ID":"3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9","Type":"ContainerDied","Data":"55f7694821f8c6081cfe230c62462e45987f3b97269c1333a6c6ea0ed461967a"}
Sep 18 20:22:20.456371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.456352    2564 scope.go:117] "RemoveContainer" containerID="8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"
Sep 18 20:22:20.465220 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.465197    2564 scope.go:117] "RemoveContainer" containerID="8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"
Sep 18 20:22:20.465465 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:22:20.465449    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf\": container with ID starting with 8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf not found: ID does not exist" containerID="8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"
Sep 18 20:22:20.465514 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.465473    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf"} err="failed to get container status \"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf\": rpc error: code = NotFound desc = could not find container \"8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf\": container with ID starting with 8429ff61f60d1bf94ce6d266c79dc20b24269b62ce2789a955cbe1058d6d0ebf not found: ID does not exist"
Sep 18 20:22:20.478174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.478149    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:22:20.481066 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:20.481048    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-976b66477-phssh"]
Sep 18 20:22:21.792483 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:21.792451    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" path="/var/lib/kubelet/pods/3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9/volumes"
Sep 18 20:22:33.342351 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:33.342310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:22:33.346083 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:22:33.346056    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d8f95559c-qslpk"
Sep 18 20:23:12.791097 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.791020    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"]
Sep 18 20:23:12.791597 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.791310    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" containerName="console"
Sep 18 20:23:12.791597 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.791321    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" containerName="console"
Sep 18 20:23:12.791597 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.791373    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3f5d8a-c5ad-488a-8ea9-41ecb2dabbb9" containerName="console"
Sep 18 20:23:12.794414 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.794395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.798179 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.798152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 20:23:12.798274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.798234    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 20:23:12.801233 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.801212    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 20:23:12.801320 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.801303    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-f7p9c\""
Sep 18 20:23:12.801430 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.801407    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 20:23:12.801921 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.801905    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 20:23:12.808228 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.808210    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 20:23:12.819181 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.819150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"]
Sep 18 20:23:12.909020 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.908983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrsfl\" (UniqueName: \"kubernetes.io/projected/8d165f96-7aaf-4458-824e-b328e4f1f6c8-kube-api-access-nrsfl\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909020 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-federate-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-serving-certs-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-metrics-client-ca\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:12.909331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:12.909274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.009898 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.009861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-serving-certs-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.009898 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.009898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-metrics-client-ca\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010122 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.009924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010122 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010122 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrsfl\" (UniqueName: \"kubernetes.io/projected/8d165f96-7aaf-4458-824e-b328e4f1f6c8-kube-api-access-nrsfl\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010122 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010319 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-federate-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010319 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010700 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-serving-certs-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.010989 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.010964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-metrics-client-ca\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.011087 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.011044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.012701 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.012673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-federate-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.012701 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.012686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-telemeter-client-tls\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.012887 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.012870    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.012925 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.012886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8d165f96-7aaf-4458-824e-b328e4f1f6c8-secret-telemeter-client\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.019596 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.019572    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrsfl\" (UniqueName: \"kubernetes.io/projected/8d165f96-7aaf-4458-824e-b328e4f1f6c8-kube-api-access-nrsfl\") pod \"telemeter-client-67b9dcc4cf-dc6sv\" (UID: \"8d165f96-7aaf-4458-824e-b328e4f1f6c8\") " pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.103754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.103652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"
Sep 18 20:23:13.381953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.381880    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv"]
Sep 18 20:23:13.385158 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:23:13.385131    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d165f96_7aaf_4458_824e_b328e4f1f6c8.slice/crio-1dfa258cdabe05e7492a927e3700345bbf4cb07a1d5d33937218ff6bacf6752d WatchSource:0}: Error finding container 1dfa258cdabe05e7492a927e3700345bbf4cb07a1d5d33937218ff6bacf6752d: Status 404 returned error can't find the container with id 1dfa258cdabe05e7492a927e3700345bbf4cb07a1d5d33937218ff6bacf6752d
Sep 18 20:23:13.661130 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:13.661043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv" event={"ID":"8d165f96-7aaf-4458-824e-b328e4f1f6c8","Type":"ContainerStarted","Data":"1dfa258cdabe05e7492a927e3700345bbf4cb07a1d5d33937218ff6bacf6752d"}
Sep 18 20:23:15.673864 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.673820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv" event={"ID":"8d165f96-7aaf-4458-824e-b328e4f1f6c8","Type":"ContainerStarted","Data":"df9ba80cb5189cf3a0c6b93ed0f6af8a2e42038cab869dfea961073178213896"}
Sep 18 20:23:15.674230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.673880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv" event={"ID":"8d165f96-7aaf-4458-824e-b328e4f1f6c8","Type":"ContainerStarted","Data":"f9328a2547cd483ffaf404426a94bab259fff23d3e9525bb3439304cb78e12a9"}
Sep 18 20:23:15.674230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.673899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv" event={"ID":"8d165f96-7aaf-4458-824e-b328e4f1f6c8","Type":"ContainerStarted","Data":"1750a5be6dbdff7d6d483b1483dfc293cd7ca5d2f2baef8bb06cf0fd5fde4a02"}
Sep 18 20:23:15.702082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.702035    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-67b9dcc4cf-dc6sv" podStartSLOduration=1.935303791 podStartE2EDuration="3.702020125s" totalImagesPullingTime="1.766716334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:15.699251357 +0000 UTC m=+168.048523750" watchObservedRunningTime="2026-09-18 20:23:15.702020125 +0000 UTC m=+168.051292517"
Sep 18 20:23:15.793444 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.793418    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:23:15.796972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.796951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.800435 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.800414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 20:23:15.800733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.800718    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-88xm9\""
Sep 18 20:23:15.800807 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.800751    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 20:23:15.801168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.801135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:23:15.801168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.801149    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 20:23:15.801324 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.801202    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 20:23:15.801601 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.801579    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:23:15.801878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.801859    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 20:23:15.810107 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.810081    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 20:23:15.810748 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.810726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:23:15.832791 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.832892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw78q\" (UniqueName: \"kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.832892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.832958 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.832995 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.832995 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.833057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.832998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.933953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.933986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934025 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw78q\" (UniqueName: \"kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934748 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934848 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934791    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934915 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.934955 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.934916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.937156 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.937129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.937268 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.937211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:15.943343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:15.943323    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw78q\" (UniqueName: \"kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q\") pod \"console-66b664bf6c-tknfc\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") " pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:16.107886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:16.107851    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:16.383343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:16.382967    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:23:16.385767 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:23:16.385735    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96447aaa_e0ce_4465_9329_edb6d4674f26.slice/crio-de687da7f8034daa9c751fff232f6b02b68303266a36f7d2119bd8acbc87a604 WatchSource:0}: Error finding container de687da7f8034daa9c751fff232f6b02b68303266a36f7d2119bd8acbc87a604: Status 404 returned error can't find the container with id de687da7f8034daa9c751fff232f6b02b68303266a36f7d2119bd8acbc87a604
Sep 18 20:23:16.679450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:16.679359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b664bf6c-tknfc" event={"ID":"96447aaa-e0ce-4465-9329-edb6d4674f26","Type":"ContainerStarted","Data":"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"}
Sep 18 20:23:16.679450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:16.679418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b664bf6c-tknfc" event={"ID":"96447aaa-e0ce-4465-9329-edb6d4674f26","Type":"ContainerStarted","Data":"de687da7f8034daa9c751fff232f6b02b68303266a36f7d2119bd8acbc87a604"}
Sep 18 20:23:16.703373 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:16.703331    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b664bf6c-tknfc" podStartSLOduration=1.703314217 podStartE2EDuration="1.703314217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:16.702047636 +0000 UTC m=+169.051320061" watchObservedRunningTime="2026-09-18 20:23:16.703314217 +0000 UTC m=+169.052586609"
Sep 18 20:23:26.108955 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.108925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:26.109413 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.108976    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:26.113664 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.113620    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:26.725514 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.725487    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:23:26.905052 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.905020    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:23:26.908365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.908351    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916391 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916361    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn98v\" (UniqueName: \"kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916788 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.916788 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.916621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:26.918000 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:26.917982    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:23:27.017726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn98v\" (UniqueName: \"kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.017980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.017807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.018474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.018444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.018570 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.018484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.018608 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.018561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.018608 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.018582    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.020716 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.020695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.020788 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.020714    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.025405 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.025383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn98v\" (UniqueName: \"kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v\") pod \"console-cdc5cb49f-qvzzd\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") " pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.218078 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.218037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:27.498211 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.498045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:23:27.501044 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:23:27.501015    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde3ee05_0517_44c0_862e_dd85f0603854.slice/crio-7353616b63de2fc436a0eaed7e2c441d1c1a49e52b32a6c0ce6dba8cf9ba654e WatchSource:0}: Error finding container 7353616b63de2fc436a0eaed7e2c441d1c1a49e52b32a6c0ce6dba8cf9ba654e: Status 404 returned error can't find the container with id 7353616b63de2fc436a0eaed7e2c441d1c1a49e52b32a6c0ce6dba8cf9ba654e
Sep 18 20:23:27.727049 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.727012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc5cb49f-qvzzd" event={"ID":"bde3ee05-0517-44c0-862e-dd85f0603854","Type":"ContainerStarted","Data":"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"}
Sep 18 20:23:27.727212 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.727065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc5cb49f-qvzzd" event={"ID":"bde3ee05-0517-44c0-862e-dd85f0603854","Type":"ContainerStarted","Data":"7353616b63de2fc436a0eaed7e2c441d1c1a49e52b32a6c0ce6dba8cf9ba654e"}
Sep 18 20:23:27.745948 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:27.745902    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cdc5cb49f-qvzzd" podStartSLOduration=1.745887743 podStartE2EDuration="1.745887743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:27.744552889 +0000 UTC m=+180.093825306" watchObservedRunningTime="2026-09-18 20:23:27.745887743 +0000 UTC m=+180.095160134"
Sep 18 20:23:37.218171 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:37.218136    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:37.218604 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:37.218201    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:37.222899 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:37.222876    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:37.774898 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:37.774867    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:23:37.826823 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:23:37.826797    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:24:02.845906 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:02.845843    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66b664bf6c-tknfc" podUID="96447aaa-e0ce-4465-9329-edb6d4674f26" containerName="console" containerID="cri-o://959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809" gracePeriod=15
Sep 18 20:24:03.082938 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.082914    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:24:03.216354 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216265    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216354 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216309    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216354 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216340    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216354 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216358    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216779 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216377    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw78q\" (UniqueName: \"kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216779 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216779 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216574    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config\") pod \"96447aaa-e0ce-4465-9329-edb6d4674f26\" (UID: \"96447aaa-e0ce-4465-9329-edb6d4674f26\") "
Sep 18 20:24:03.216935 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216837    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca" (OuterVolumeSpecName: "service-ca") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:03.216935 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.216855    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:03.217050 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.217024    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:03.217117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.217098    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config" (OuterVolumeSpecName: "console-config") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:03.218585 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.218562    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:24:03.218690 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.218595    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q" (OuterVolumeSpecName: "kube-api-access-sw78q") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "kube-api-access-sw78q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:03.218739 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.218684    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "96447aaa-e0ce-4465-9329-edb6d4674f26" (UID: "96447aaa-e0ce-4465-9329-edb6d4674f26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:24:03.317999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.317950    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-trusted-ca-bundle\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.317999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.317996    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-oauth-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.317999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.318007    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96447aaa-e0ce-4465-9329-edb6d4674f26-console-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.317999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.318016    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-service-ca\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.318277 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.318025    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw78q\" (UniqueName: \"kubernetes.io/projected/96447aaa-e0ce-4465-9329-edb6d4674f26-kube-api-access-sw78q\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.318277 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.318034    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-oauth-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.318277 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.318042    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96447aaa-e0ce-4465-9329-edb6d4674f26-console-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.871917 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.871882    2564 generic.go:356] "Generic (PLEG): container finished" podID="96447aaa-e0ce-4465-9329-edb6d4674f26" containerID="959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809" exitCode=0
Sep 18 20:24:03.872334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.871926    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b664bf6c-tknfc"
Sep 18 20:24:03.872334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.871967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b664bf6c-tknfc" event={"ID":"96447aaa-e0ce-4465-9329-edb6d4674f26","Type":"ContainerDied","Data":"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"}
Sep 18 20:24:03.872334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.872016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b664bf6c-tknfc" event={"ID":"96447aaa-e0ce-4465-9329-edb6d4674f26","Type":"ContainerDied","Data":"de687da7f8034daa9c751fff232f6b02b68303266a36f7d2119bd8acbc87a604"}
Sep 18 20:24:03.872334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.872039    2564 scope.go:117] "RemoveContainer" containerID="959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"
Sep 18 20:24:03.879940 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.879924    2564 scope.go:117] "RemoveContainer" containerID="959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"
Sep 18 20:24:03.880202 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:24:03.880185    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809\": container with ID starting with 959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809 not found: ID does not exist" containerID="959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"
Sep 18 20:24:03.880251 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.880216    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809"} err="failed to get container status \"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809\": rpc error: code = NotFound desc = could not find container \"959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809\": container with ID starting with 959a606352211dc363c25fab65fc218c30b055f6553483c67c5f592925889809 not found: ID does not exist"
Sep 18 20:24:03.895034 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.895010    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:24:03.899529 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:03.899508    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b664bf6c-tknfc"]
Sep 18 20:24:05.792706 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:24:05.792672    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96447aaa-e0ce-4465-9329-edb6d4674f26" path="/var/lib/kubelet/pods/96447aaa-e0ce-4465-9329-edb6d4674f26/volumes"
Sep 18 20:25:29.737031 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:29.737000    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:25:29.740455 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:29.740430    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 20:25:29.745746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:29.745725    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:25:37.581346 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.581312    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-r4ckt"]
Sep 18 20:25:37.583655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.581604    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96447aaa-e0ce-4465-9329-edb6d4674f26" containerName="console"
Sep 18 20:25:37.583655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.581615    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96447aaa-e0ce-4465-9329-edb6d4674f26" containerName="console"
Sep 18 20:25:37.583655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.581691    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96447aaa-e0ce-4465-9329-edb6d4674f26" containerName="console"
Sep 18 20:25:37.584552 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.584537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.587426 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.587406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 18 20:25:37.588840 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.588819    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 18 20:25:37.588942 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.588864    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 18 20:25:37.588942 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.588877    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-pv8jd\""
Sep 18 20:25:37.589133 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.589099    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 18 20:25:37.606740 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.606714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-r4ckt"]
Sep 18 20:25:37.673217 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.673181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-certificates\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.673217 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.673222    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd62r\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-kube-api-access-rd62r\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.774495 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.774466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-certificates\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.774673 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.774508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd62r\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-kube-api-access-rd62r\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.776922 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.776894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-certificates\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.788313 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.788287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd62r\" (UniqueName: \"kubernetes.io/projected/952749be-052b-4838-bc73-da83e3762204-kube-api-access-rd62r\") pod \"keda-admission-6c4d864b86-r4ckt\" (UID: \"952749be-052b-4838-bc73-da83e3762204\") " pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:37.896495 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:37.896405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:38.175119 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:38.175098    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-r4ckt"]
Sep 18 20:25:38.177304 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:25:38.177273    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952749be_052b_4838_bc73_da83e3762204.slice/crio-183fdb0513f5cf4a20bb83772fc3da438d8821da3fb54fdec03fbfa579db64c9 WatchSource:0}: Error finding container 183fdb0513f5cf4a20bb83772fc3da438d8821da3fb54fdec03fbfa579db64c9: Status 404 returned error can't find the container with id 183fdb0513f5cf4a20bb83772fc3da438d8821da3fb54fdec03fbfa579db64c9
Sep 18 20:25:38.178642 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:38.178615    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:25:38.228641 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:38.228588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt" event={"ID":"952749be-052b-4838-bc73-da83e3762204","Type":"ContainerStarted","Data":"183fdb0513f5cf4a20bb83772fc3da438d8821da3fb54fdec03fbfa579db64c9"}
Sep 18 20:25:41.244754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:41.244712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt" event={"ID":"952749be-052b-4838-bc73-da83e3762204","Type":"ContainerStarted","Data":"ee74921729dd7e71b11dfa5665ece244716063479962202d4ebae5efa52dd6ff"}
Sep 18 20:25:41.245173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:41.244824    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:25:41.267142 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:25:41.267097    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt" podStartSLOduration=1.723795226 podStartE2EDuration="4.26708512s" totalImagesPullingTime="2.543289894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:25:41.265176684 +0000 UTC m=+313.614449100" watchObservedRunningTime="2026-09-18 20:25:41.26708512 +0000 UTC m=+313.616357512"
Sep 18 20:26:02.250421 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:02.250389    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-r4ckt"
Sep 18 20:26:38.520590 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.520560    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85bf7d6cd4-v59zf"]
Sep 18 20:26:38.523813 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.523797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.526505 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:26:38.526478    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-85bf7d6cd4-v59zf\" is forbidden: User \"system:node:ip-10-0-140-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-140-180.ec2.internal' and this object" podUID="5e9a4603-c0b5-43f2-a118-e6ebd38287ab" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.554951 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.554928    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bf7d6cd4-v59zf"]
Sep 18 20:26:38.617832 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-oauth-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.617953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-service-ca\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.617953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617861    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-oauth-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.617953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.617953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-trusted-ca-bundle\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.618111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.617996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lvzb\" (UniqueName: \"kubernetes.io/projected/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-kube-api-access-9lvzb\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.618111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.618027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.718889 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.718857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-service-ca\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.719026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.718898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-oauth-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.719026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.718940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.719026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.718974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-trusted-ca-bundle\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.719026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.719007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lvzb\" (UniqueName: \"kubernetes.io/projected/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-kube-api-access-9lvzb\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.719995 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.719962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-oauth-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.720168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.720127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-service-ca\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.720168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.720093    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-trusted-ca-bundle\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.720330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.719167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.720330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.720252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.720330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.720296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-oauth-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.722546 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.722525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-serving-cert\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.723124 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.723099    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-console-oauth-config\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.728199 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.728178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lvzb\" (UniqueName: \"kubernetes.io/projected/5e9a4603-c0b5-43f2-a118-e6ebd38287ab-kube-api-access-9lvzb\") pod \"console-85bf7d6cd4-v59zf\" (UID: \"5e9a4603-c0b5-43f2-a118-e6ebd38287ab\") " pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:38.832946 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:38.832923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:39.101746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:39.101723    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bf7d6cd4-v59zf"]
Sep 18 20:26:39.103705 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:26:39.103674    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9a4603_c0b5_43f2_a118_e6ebd38287ab.slice/crio-ff8849042b46cc6f254c820153868038eab687d5955d3dbac119f6ff4409c13b WatchSource:0}: Error finding container ff8849042b46cc6f254c820153868038eab687d5955d3dbac119f6ff4409c13b: Status 404 returned error can't find the container with id ff8849042b46cc6f254c820153868038eab687d5955d3dbac119f6ff4409c13b
Sep 18 20:26:39.474570 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:39.474484    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf7d6cd4-v59zf" event={"ID":"5e9a4603-c0b5-43f2-a118-e6ebd38287ab","Type":"ContainerStarted","Data":"f5b1ec45f5651285222bc00b9723996860cfd8d7c27663daae6225f2746b218b"}
Sep 18 20:26:39.474570 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:39.474528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf7d6cd4-v59zf" event={"ID":"5e9a4603-c0b5-43f2-a118-e6ebd38287ab","Type":"ContainerStarted","Data":"ff8849042b46cc6f254c820153868038eab687d5955d3dbac119f6ff4409c13b"}
Sep 18 20:26:39.497771 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:39.497727    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85bf7d6cd4-v59zf" podStartSLOduration=1.497712279 podStartE2EDuration="1.497712279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:26:39.49550332 +0000 UTC m=+371.844775713" watchObservedRunningTime="2026-09-18 20:26:39.497712279 +0000 UTC m=+371.846984672"
Sep 18 20:26:48.833285 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:48.833250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:48.833747 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:48.833324    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:48.837496 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:48.837476    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:49.515423 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:49.515371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85bf7d6cd4-v59zf"
Sep 18 20:26:49.580222 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:26:49.580194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:27:02.799529 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.799498    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:02.802936 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.802916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:02.806889 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.806868    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-jdp8j\""
Sep 18 20:27:02.807343 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.807327    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 18 20:27:02.808149 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.808132    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 18 20:27:02.808236 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.808162    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 18 20:27:02.825562 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.825535    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:02.916505 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.916468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nj96\" (UniqueName: \"kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:02.916711 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:02.916599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.017972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.017940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nj96\" (UniqueName: \"kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.018152 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.018011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.020476 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.020449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.027675 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.027649    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nj96\" (UniqueName: \"kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96\") pod \"kserve-controller-manager-7dcf4bff6-hgb67\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") " pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.116688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.116581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:03.394355 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.394322    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:03.394924 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:27:03.394903    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9cde2a_76c0_4c25_b4e0_8113febdeb5f.slice/crio-963dcaa8abd8a291ea036b3a4ada6f5e2f4cf3d7f4e5204721830289237da1d3 WatchSource:0}: Error finding container 963dcaa8abd8a291ea036b3a4ada6f5e2f4cf3d7f4e5204721830289237da1d3: Status 404 returned error can't find the container with id 963dcaa8abd8a291ea036b3a4ada6f5e2f4cf3d7f4e5204721830289237da1d3
Sep 18 20:27:03.565312 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:03.565271    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" event={"ID":"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f","Type":"ContainerStarted","Data":"963dcaa8abd8a291ea036b3a4ada6f5e2f4cf3d7f4e5204721830289237da1d3"}
Sep 18 20:27:06.580927 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:06.580890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" event={"ID":"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f","Type":"ContainerStarted","Data":"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"}
Sep 18 20:27:06.581287 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:06.581017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:06.598979 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:06.598936    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" podStartSLOduration=2.196150389 podStartE2EDuration="4.598921922s" totalImagesPullingTime="2.402771533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:27:06.59754362 +0000 UTC m=+398.946816014" watchObservedRunningTime="2026-09-18 20:27:06.598921922 +0000 UTC m=+398.948194315"
Sep 18 20:27:14.599186 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:14.599109    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cdc5cb49f-qvzzd" podUID="bde3ee05-0517-44c0-862e-dd85f0603854" containerName="console" containerID="cri-o://21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9" gracePeriod=15
Sep 18 20:27:14.846836 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:14.846812    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:27:15.011639 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011611    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.011756 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011668    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.011857 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn98v\" (UniqueName: \"kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.011910 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011897    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.011986 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011971    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.012028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.011989    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:15.012072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012031    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config" (OuterVolumeSpecName: "console-config") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:15.012072 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012058    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.012153 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012091    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config\") pod \"bde3ee05-0517-44c0-862e-dd85f0603854\" (UID: \"bde3ee05-0517-44c0-862e-dd85f0603854\") "
Sep 18 20:27:15.012323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012302    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:15.012413 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012315    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca" (OuterVolumeSpecName: "service-ca") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:15.012413 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012309    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-trusted-ca-bundle\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.012413 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.012370    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-console-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.013979 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.013956    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v" (OuterVolumeSpecName: "kube-api-access-hn98v") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "kube-api-access-hn98v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:15.014102 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.014084    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:15.014266 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.014245    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bde3ee05-0517-44c0-862e-dd85f0603854" (UID: "bde3ee05-0517-44c0-862e-dd85f0603854"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:15.113809 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.113756    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.113809 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.113803    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde3ee05-0517-44c0-862e-dd85f0603854-console-oauth-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.113809 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.113816    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn98v\" (UniqueName: \"kubernetes.io/projected/bde3ee05-0517-44c0-862e-dd85f0603854-kube-api-access-hn98v\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.114085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.113829    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-service-ca\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.114085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.113842    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde3ee05-0517-44c0-862e-dd85f0603854-oauth-serving-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:15.617786 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.617751    2564 generic.go:356] "Generic (PLEG): container finished" podID="bde3ee05-0517-44c0-862e-dd85f0603854" containerID="21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9" exitCode=0
Sep 18 20:27:15.618180 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.617796    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc5cb49f-qvzzd"
Sep 18 20:27:15.618180 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.617818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc5cb49f-qvzzd" event={"ID":"bde3ee05-0517-44c0-862e-dd85f0603854","Type":"ContainerDied","Data":"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"}
Sep 18 20:27:15.618180 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.617898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc5cb49f-qvzzd" event={"ID":"bde3ee05-0517-44c0-862e-dd85f0603854","Type":"ContainerDied","Data":"7353616b63de2fc436a0eaed7e2c441d1c1a49e52b32a6c0ce6dba8cf9ba654e"}
Sep 18 20:27:15.618180 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.617923    2564 scope.go:117] "RemoveContainer" containerID="21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"
Sep 18 20:27:15.626304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.626284    2564 scope.go:117] "RemoveContainer" containerID="21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"
Sep 18 20:27:15.626544 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:27:15.626526    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9\": container with ID starting with 21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9 not found: ID does not exist" containerID="21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"
Sep 18 20:27:15.626603 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.626552    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9"} err="failed to get container status \"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9\": rpc error: code = NotFound desc = could not find container \"21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9\": container with ID starting with 21d97e33c997466b4f35098a343687146649edb528d02c97df5083cd8d0d7bd9 not found: ID does not exist"
Sep 18 20:27:15.644430 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.644410    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:27:15.648124 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.648100    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cdc5cb49f-qvzzd"]
Sep 18 20:27:15.793753 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:15.793719    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bde3ee05-0517-44c0-862e-dd85f0603854" path="/var/lib/kubelet/pods/bde3ee05-0517-44c0-862e-dd85f0603854/volumes"
Sep 18 20:27:37.589367 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:37.589325    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:48.313724 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.313646    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:48.314101 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.313856    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" podUID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" containerName="manager" containerID="cri-o://2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969" gracePeriod=10
Sep 18 20:27:48.359301 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.359265    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-zphhn"]
Sep 18 20:27:48.359601 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.359590    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bde3ee05-0517-44c0-862e-dd85f0603854" containerName="console"
Sep 18 20:27:48.359655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.359602    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bde3ee05-0517-44c0-862e-dd85f0603854" containerName="console"
Sep 18 20:27:48.359692 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.359675    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bde3ee05-0517-44c0-862e-dd85f0603854" containerName="console"
Sep 18 20:27:48.362529 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.362513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.381769 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.381738    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-zphhn"]
Sep 18 20:27:48.465993 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.465960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xjj6\" (UniqueName: \"kubernetes.io/projected/a2aedfb4-48bb-4b1d-b415-b25361564cbb-kube-api-access-8xjj6\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.466149 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.466053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2aedfb4-48bb-4b1d-b415-b25361564cbb-cert\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.560284 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.560254    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:48.566824 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.566767    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xjj6\" (UniqueName: \"kubernetes.io/projected/a2aedfb4-48bb-4b1d-b415-b25361564cbb-kube-api-access-8xjj6\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.566932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.566853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2aedfb4-48bb-4b1d-b415-b25361564cbb-cert\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.569148 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.569126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2aedfb4-48bb-4b1d-b415-b25361564cbb-cert\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.577359 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.577330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xjj6\" (UniqueName: \"kubernetes.io/projected/a2aedfb4-48bb-4b1d-b415-b25361564cbb-kube-api-access-8xjj6\") pod \"kserve-controller-manager-7dcf4bff6-zphhn\" (UID: \"a2aedfb4-48bb-4b1d-b415-b25361564cbb\") " pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.668206 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.668174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert\") pod \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") "
Sep 18 20:27:48.668365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.668272    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nj96\" (UniqueName: \"kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96\") pod \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\" (UID: \"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f\") "
Sep 18 20:27:48.670389 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.670359    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert" (OuterVolumeSpecName: "cert") pod "dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" (UID: "dc9cde2a-76c0-4c25-b4e0-8113febdeb5f"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:48.670499 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.670401    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96" (OuterVolumeSpecName: "kube-api-access-4nj96") pod "dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" (UID: "dc9cde2a-76c0-4c25-b4e0-8113febdeb5f"). InnerVolumeSpecName "kube-api-access-4nj96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:48.699585 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.699554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:48.751412 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.751380    2564 generic.go:356] "Generic (PLEG): container finished" podID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" containerID="2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969" exitCode=0
Sep 18 20:27:48.751532 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.751459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" event={"ID":"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f","Type":"ContainerDied","Data":"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"}
Sep 18 20:27:48.751532 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.751516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67" event={"ID":"dc9cde2a-76c0-4c25-b4e0-8113febdeb5f","Type":"ContainerDied","Data":"963dcaa8abd8a291ea036b3a4ada6f5e2f4cf3d7f4e5204721830289237da1d3"}
Sep 18 20:27:48.751670 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.751548    2564 scope.go:117] "RemoveContainer" containerID="2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"
Sep 18 20:27:48.752724 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.752040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-hgb67"
Sep 18 20:27:48.763700 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.763676    2564 scope.go:117] "RemoveContainer" containerID="2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"
Sep 18 20:27:48.764351 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:27:48.764328    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969\": container with ID starting with 2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969 not found: ID does not exist" containerID="2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"
Sep 18 20:27:48.764424 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.764365    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969"} err="failed to get container status \"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969\": rpc error: code = NotFound desc = could not find container \"2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969\": container with ID starting with 2a489ae41ca49c81c3dd40cfdf93f7053b8cdc6b41e3e4e077d5cfb43eeb2969 not found: ID does not exist"
Sep 18 20:27:48.769125 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.769098    2564 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:48.769225 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.769130    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nj96\" (UniqueName: \"kubernetes.io/projected/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f-kube-api-access-4nj96\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:27:48.781049 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.781013    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:48.786027 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.786004    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-hgb67"]
Sep 18 20:27:48.972620 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:48.972594    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-zphhn"]
Sep 18 20:27:48.974115 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:27:48.974089    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2aedfb4_48bb_4b1d_b415_b25361564cbb.slice/crio-f6b6a81b1ed2984bb9c8febf154c3247937728e2462af980c504178ea4a2bffe WatchSource:0}: Error finding container f6b6a81b1ed2984bb9c8febf154c3247937728e2462af980c504178ea4a2bffe: Status 404 returned error can't find the container with id f6b6a81b1ed2984bb9c8febf154c3247937728e2462af980c504178ea4a2bffe
Sep 18 20:27:49.756708 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:49.756602    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn" event={"ID":"a2aedfb4-48bb-4b1d-b415-b25361564cbb","Type":"ContainerStarted","Data":"71ce326f8d1e224dba08b94e5ec011c9a3fcecc36a7327f2f419668e047f13c0"}
Sep 18 20:27:49.756708 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:49.756692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:27:49.756708 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:49.756711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn" event={"ID":"a2aedfb4-48bb-4b1d-b415-b25361564cbb","Type":"ContainerStarted","Data":"f6b6a81b1ed2984bb9c8febf154c3247937728e2462af980c504178ea4a2bffe"}
Sep 18 20:27:49.792977 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:27:49.792942    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" path="/var/lib/kubelet/pods/dc9cde2a-76c0-4c25-b4e0-8113febdeb5f/volumes"
Sep 18 20:28:20.766775 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:20.766742    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn"
Sep 18 20:28:20.786048 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:20.785981    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7dcf4bff6-zphhn" podStartSLOduration=32.379058552 podStartE2EDuration="32.785965321s" totalImagesPullingTime="406.906769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:27:49.781812123 +0000 UTC m=+442.131084549" watchObservedRunningTime="2026-09-18 20:28:20.785965321 +0000 UTC m=+473.135237714"
Sep 18 20:28:38.267120 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.267090    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-nmvrm"]
Sep 18 20:28:38.267473 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.267408    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" containerName="manager"
Sep 18 20:28:38.267473 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.267418    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" containerName="manager"
Sep 18 20:28:38.267544 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.267484    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9cde2a-76c0-4c25-b4e0-8113febdeb5f" containerName="manager"
Sep 18 20:28:38.271706 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.271690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.274646 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.274617    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 18 20:28:38.274866 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.274853    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Sep 18 20:28:38.275248 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.275236    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-7klnw\""
Sep 18 20:28:38.282242 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.282223    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-nmvrm"]
Sep 18 20:28:38.368600 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.368574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.368745 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.368621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97h2k\" (UniqueName: \"kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.469826 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.469802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.469916 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.469842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97h2k\" (UniqueName: \"kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.470171 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.470151    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.478365 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.478344    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97h2k\" (UniqueName: \"kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k\") pod \"s3-init-nmvrm\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") " pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.580536 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.580515    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-nmvrm"
Sep 18 20:28:38.847964 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.847896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-nmvrm"]
Sep 18 20:28:38.850828 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:28:38.850803    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dccb575_3dea_4a09_858b_5e64cc905e47.slice/crio-eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62 WatchSource:0}: Error finding container eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62: Status 404 returned error can't find the container with id eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62
Sep 18 20:28:38.948886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:38.948861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerStarted","Data":"eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62"}
Sep 18 20:28:41.965952 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:41.965915    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerStarted","Data":"a8fca96b80fc22f2c73773c174fd7f157d897d899127d7ecf7a0062d2332219c"}
Sep 18 20:28:43.976775 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:43.976701    2564 generic.go:356] "Generic (PLEG): container finished" podID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerID="a8fca96b80fc22f2c73773c174fd7f157d897d899127d7ecf7a0062d2332219c" exitCode=0
Sep 18 20:28:43.976775 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:43.976751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerDied","Data":"a8fca96b80fc22f2c73773c174fd7f157d897d899127d7ecf7a0062d2332219c"}
Sep 18 20:28:49.002912 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:49.002879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerStarted","Data":"c37421adb44484569e2418a55afdc20eb11b9c282421800d8ee839d19e70b5b5"}
Sep 18 20:28:49.022383 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:28:49.022336    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-nmvrm" podStartSLOduration=1.877483357 podStartE2EDuration="11.022323262s" totalImagesPullingTime="7.350950939s" totalInitContainerRuntime="1.793888966s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:49.021249058 +0000 UTC m=+501.370521452" watchObservedRunningTime="2026-09-18 20:28:49.022323262 +0000 UTC m=+501.371595654"
Sep 18 20:29:54.266064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:54.266028    2564 generic.go:356] "Generic (PLEG): container finished" podID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerID="c37421adb44484569e2418a55afdc20eb11b9c282421800d8ee839d19e70b5b5" exitCode=0
Sep 18 20:29:54.266479 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:54.266101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerDied","Data":"c37421adb44484569e2418a55afdc20eb11b9c282421800d8ee839d19e70b5b5"}
Sep 18 20:29:55.393058 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.393033    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-nmvrm"
Sep 18 20:29:55.499039 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.499002    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97h2k\" (UniqueName: \"kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k\") pod \"1dccb575-3dea-4a09-858b-5e64cc905e47\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") "
Sep 18 20:29:55.499229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.499083    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache\") pod \"1dccb575-3dea-4a09-858b-5e64cc905e47\" (UID: \"1dccb575-3dea-4a09-858b-5e64cc905e47\") "
Sep 18 20:29:55.501092 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.501061    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k" (OuterVolumeSpecName: "kube-api-access-97h2k") pod "1dccb575-3dea-4a09-858b-5e64cc905e47" (UID: "1dccb575-3dea-4a09-858b-5e64cc905e47"). InnerVolumeSpecName "kube-api-access-97h2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:55.521656 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.521582    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache" (OuterVolumeSpecName: "model-cache") pod "1dccb575-3dea-4a09-858b-5e64cc905e47" (UID: "1dccb575-3dea-4a09-858b-5e64cc905e47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:55.600641 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.600602    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dccb575-3dea-4a09-858b-5e64cc905e47-model-cache\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:29:55.600754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:55.600653    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97h2k\" (UniqueName: \"kubernetes.io/projected/1dccb575-3dea-4a09-858b-5e64cc905e47-kube-api-access-97h2k\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:29:56.276121 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:56.276031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-nmvrm" event={"ID":"1dccb575-3dea-4a09-858b-5e64cc905e47","Type":"ContainerDied","Data":"eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62"}
Sep 18 20:29:56.276121 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:56.276058    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-nmvrm"
Sep 18 20:29:56.276121 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:29:56.276077    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb8f911396982e58f04c3885603f02b08a0c3c20d0dff39ea6285ebdfc524b62"
Sep 18 20:30:09.035756 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.035724    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:30:09.036109 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.036081    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerName="s3-init"
Sep 18 20:30:09.036109 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.036095    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerName="s3-init"
Sep 18 20:30:09.036178 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.036117    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerName="download-hf-model"
Sep 18 20:30:09.036178 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.036123    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerName="download-hf-model"
Sep 18 20:30:09.036294 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.036246    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dccb575-3dea-4a09-858b-5e64cc905e47" containerName="s3-init"
Sep 18 20:30:09.039654 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.039625    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.042603 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.042586    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 18 20:30:09.042717 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.042606    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 18 20:30:09.042717 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.042708    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Sep 18 20:30:09.043883 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.043861    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 18 20:30:09.043954 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.043910    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6hbfs\""
Sep 18 20:30:09.044009 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.043971    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 18 20:30:09.044065 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.043971    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Sep 18 20:30:09.049445 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.049422    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:30:09.122126 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.122090    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:30:09.125366 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.125351    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.128317 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.128288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ef587-predictor-serving-cert\""
Sep 18 20:30:09.128420 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.128340    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ef587-kube-rbac-proxy-sar-config\""
Sep 18 20:30:09.135069 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.135047    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:30:09.212377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.212377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffttj\" (UniqueName: \"kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.212607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.212607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.212607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.212607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212573    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.212607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.212932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.212618    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwdxr\" (UniqueName: \"kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.257537 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.257508    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:30:09.261509 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.261491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.264162 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.264143    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Sep 18 20:30:09.264251 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.264170    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Sep 18 20:30:09.273900 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.273874    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:30:09.313330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.313330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.313330 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwdxr\" (UniqueName: \"kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.313567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.313567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffttj\" (UniqueName: \"kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.313567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.313567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.313786 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.313744    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.314048 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.314026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.314166 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.314060    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.314166 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.314156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.314293 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.314187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.316090 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.316042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.316459 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.316436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.323303 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.323278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwdxr\" (UniqueName: \"kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr\") pod \"success-200-isvc-ef587-predictor-6d4f85455-lpmjf\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") " pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.323700 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.323679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffttj\" (UniqueName: \"kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj\") pod \"isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.350724 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.350703    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:09.415221 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.415183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfpmv\" (UniqueName: \"kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.415746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.415290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.415746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.415367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.415746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.415404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.415746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.415497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.436189 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.436167    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:09.517274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.517077    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.517274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.517198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.517274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.517240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.517507 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.517294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.517507 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.517358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfpmv\" (UniqueName: \"kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.518678 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.518611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.518866 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.518825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.519132 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.519109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.521553 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.521530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.530064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.530042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfpmv\" (UniqueName: \"kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xtp66\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.544410 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.544388    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:30:09.549313 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.549299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.552124 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.552104    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Sep 18 20:30:09.552212 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.552127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Sep 18 20:30:09.561148 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.561125    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:30:09.572363 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.572313    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:09.627927 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.625648    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:30:09.629135 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:30:09.629102    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969b90fa_d180_44b9_b478_29eac744e4c0.slice/crio-85bd09f5d0539b6d9eca5505915c95d23d0532f7affbf1f5ad4ce20936f7f8a1 WatchSource:0}: Error finding container 85bd09f5d0539b6d9eca5505915c95d23d0532f7affbf1f5ad4ce20936f7f8a1: Status 404 returned error can't find the container with id 85bd09f5d0539b6d9eca5505915c95d23d0532f7affbf1f5ad4ce20936f7f8a1
Sep 18 20:30:09.717521 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.717494    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:30:09.719119 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.719096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.719214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.719124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.719214 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.719142    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.719323 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.719294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.719377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.719346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r95r9\" (UniqueName: \"kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.719488 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:30:09.719468    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0974e185_f273_43a1_9384_7c9382e2e973.slice/crio-1f25e390bd0e0c5512f1f9e00c8ceb25d1d0b08b8d0ea647e626605d3b6228bd WatchSource:0}: Error finding container 1f25e390bd0e0c5512f1f9e00c8ceb25d1d0b08b8d0ea647e626605d3b6228bd: Status 404 returned error can't find the container with id 1f25e390bd0e0c5512f1f9e00c8ceb25d1d0b08b8d0ea647e626605d3b6228bd
Sep 18 20:30:09.820045 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820045 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820274 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r95r9\" (UniqueName: \"kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820489 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820829 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.820918 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.820826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.822821 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.822768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.828035 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.828012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r95r9\" (UniqueName: \"kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9\") pod \"isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:09.844649 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.844441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:30:09.847314 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:30:09.847283    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284290e5_8a67_4df3_a30d_34d3af70b7bc.slice/crio-10bd41917aa7bef8a28e557a8ff4db02a5fad2504a7fe1aa858a982e0c72d6ea WatchSource:0}: Error finding container 10bd41917aa7bef8a28e557a8ff4db02a5fad2504a7fe1aa858a982e0c72d6ea: Status 404 returned error can't find the container with id 10bd41917aa7bef8a28e557a8ff4db02a5fad2504a7fe1aa858a982e0c72d6ea
Sep 18 20:30:09.859787 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:09.859760    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:10.138511 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.138479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:30:10.139491 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:30:10.139459    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fd09d8_0967_424a_9535_35a2ccbf896e.slice/crio-34805c33cee634bcd889db2faac47475e890f7398d8a2a982b7cef2d8ad0e82c WatchSource:0}: Error finding container 34805c33cee634bcd889db2faac47475e890f7398d8a2a982b7cef2d8ad0e82c: Status 404 returned error can't find the container with id 34805c33cee634bcd889db2faac47475e890f7398d8a2a982b7cef2d8ad0e82c
Sep 18 20:30:10.368057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.368018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerStarted","Data":"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"}
Sep 18 20:30:10.368231 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.368098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerStarted","Data":"85bd09f5d0539b6d9eca5505915c95d23d0532f7affbf1f5ad4ce20936f7f8a1"}
Sep 18 20:30:10.373615 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.373582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerStarted","Data":"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"}
Sep 18 20:30:10.373744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.373667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerStarted","Data":"10bd41917aa7bef8a28e557a8ff4db02a5fad2504a7fe1aa858a982e0c72d6ea"}
Sep 18 20:30:10.403116 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.403034    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerStarted","Data":"02703fba67f4ff35d315286f7d8d49628aeaf55e7ea388300ecda976a8367508"}
Sep 18 20:30:10.403953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.403919    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerStarted","Data":"34805c33cee634bcd889db2faac47475e890f7398d8a2a982b7cef2d8ad0e82c"}
Sep 18 20:30:10.410234 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:10.410206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerStarted","Data":"1f25e390bd0e0c5512f1f9e00c8ceb25d1d0b08b8d0ea647e626605d3b6228bd"}
Sep 18 20:30:14.441746 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:14.441708    2564 generic.go:356] "Generic (PLEG): container finished" podID="969b90fa-d180-44b9-b478-29eac744e4c0" containerID="4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d" exitCode=0
Sep 18 20:30:14.442173 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:14.441782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerDied","Data":"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"}
Sep 18 20:30:15.451168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:15.451068    2564 generic.go:356] "Generic (PLEG): container finished" podID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerID="115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db" exitCode=0
Sep 18 20:30:15.451649 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:15.451169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerDied","Data":"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"}
Sep 18 20:30:15.454074 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:15.453486    2564 generic.go:356] "Generic (PLEG): container finished" podID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerID="02703fba67f4ff35d315286f7d8d49628aeaf55e7ea388300ecda976a8367508" exitCode=0
Sep 18 20:30:15.454074 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:15.453537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerDied","Data":"02703fba67f4ff35d315286f7d8d49628aeaf55e7ea388300ecda976a8367508"}
Sep 18 20:30:27.526286 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:27.526248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerStarted","Data":"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"}
Sep 18 20:30:28.536466 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:28.536389    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerStarted","Data":"4164f82afceffa2901ba02a10c652051f0694e5c47dff201e184313cbd02cdd5"}
Sep 18 20:30:28.539593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:28.539509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerStarted","Data":"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"}
Sep 18 20:30:32.379892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.379860    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:30:32.380789 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.380769    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:30:32.570477 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.570380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerStarted","Data":"4a031f462ebf52d51f4f253f8f0e6dd36437429bac5c611de2d11d0fd96744e0"}
Sep 18 20:30:32.570908 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.570886    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:32.570975 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.570948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:32.572219 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.572183    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:30:32.573261 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.573237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerStarted","Data":"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"}
Sep 18 20:30:32.573846 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.573827    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:32.573921 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.573876    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:32.575260 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.575234    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:30:32.577068 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.577045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerStarted","Data":"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"}
Sep 18 20:30:32.577154 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.577118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:32.577217 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.577158    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:32.578174 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.578149    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:30:32.594270 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.594219    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podStartSLOduration=2.474964766 podStartE2EDuration="23.594203203s" totalImagesPullingTime="16.759770341s" totalInitContainerRuntime="4.359468096s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:32.59248949 +0000 UTC m=+604.941761885" watchObservedRunningTime="2026-09-18 20:30:32.594203203 +0000 UTC m=+604.943475596"
Sep 18 20:30:32.612119 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.611984    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podStartSLOduration=1.744757038 podStartE2EDuration="23.611965979s" totalImagesPullingTime="17.766894132s" totalInitContainerRuntime="4.100314809s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:32.611778857 +0000 UTC m=+604.961051270" watchObservedRunningTime="2026-09-18 20:30:32.611965979 +0000 UTC m=+604.961238373"
Sep 18 20:30:32.633681 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:32.633609    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podStartSLOduration=1.268859726 podStartE2EDuration="23.633589516s" totalImagesPullingTime="22.36472979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:32.632258233 +0000 UTC m=+604.981530675" watchObservedRunningTime="2026-09-18 20:30:32.633589516 +0000 UTC m=+604.982861913"
Sep 18 20:30:33.583308 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:33.583270    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:30:33.583783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:33.583267    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:30:33.583783 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:33.583281    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:30:34.588576 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:34.588541    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:30:38.588079 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:38.588050    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:30:38.590551 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:38.588770    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:30:38.590551 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:38.588806    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:30:38.590551 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:38.589214    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:30:39.593999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:39.593967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:30:39.594699 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:39.594670    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:30:48.589437 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.589396    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:30:48.589934 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.589395    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:30:48.664998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.664961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerStarted","Data":"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"}
Sep 18 20:30:48.665163 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.665024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerStarted","Data":"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"}
Sep 18 20:30:48.665367 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.665345    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:48.665501 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.665484    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:48.667021 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.666995    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:30:48.686767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:48.686711    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podStartSLOduration=3.188829506 podStartE2EDuration="39.686695037s" totalImagesPullingTime="32.39985731s" totalInitContainerRuntime="4.098008221s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:48.684254788 +0000 UTC m=+621.033527201" watchObservedRunningTime="2026-09-18 20:30:48.686695037 +0000 UTC m=+621.035967430"
Sep 18 20:30:49.595367 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:49.595329    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:30:49.670230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:49.670192    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:30:54.674722 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:54.674694    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:30:54.675241 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:54.675216    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:30:58.589605 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:58.589568    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:30:58.590080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:58.589571    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:30:59.595105 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:30:59.595067    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:31:04.675838 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:04.675793    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:08.589325 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:08.589286    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:31:08.589325 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:08.589313    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:31:09.594693 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:09.594653    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:31:14.675278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:14.675242    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:18.589169 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:18.589123    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:31:18.589612 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:18.589249    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:31:19.594966 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:19.594927    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 18 20:31:24.675712 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:24.675679    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:28.588805 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:28.588762    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:31:28.589209 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:28.589180    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:31:29.595292 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:29.595266    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:31:34.675128 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:34.675092    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:38.589237 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:38.589187    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:31:38.589654 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:38.589187    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:31:44.676060 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:44.676016    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:48.589513 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:48.589473    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 18 20:31:48.589941 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:48.589472    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 18 20:31:54.675489 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:54.675452    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:31:58.589293 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:58.589262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:31:58.589789 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:31:58.589766    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:32:00.065438 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.065405    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:32:00.065980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.065682    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container" containerID="cri-o://e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52" gracePeriod=30
Sep 18 20:32:00.065980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.065761    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kube-rbac-proxy" containerID="cri-o://74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9" gracePeriod=30
Sep 18 20:32:00.137645 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.137599    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:32:00.141335 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.141315    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.145782 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.145759    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5b4a7-predictor-serving-cert\""
Sep 18 20:32:00.145898 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.145761    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5b4a7-kube-rbac-proxy-sar-config\""
Sep 18 20:32:00.154833 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.154806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:32:00.228872 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.228834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.229062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.228938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.229062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.228991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr2sf\" (UniqueName: \"kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.329826 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.329738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr2sf\" (UniqueName: \"kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.330097 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.329903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.330097 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.329991    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.330698 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.330673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.332474 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.332447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.338752 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.338728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr2sf\" (UniqueName: \"kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf\") pod \"success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") " pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.452595 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.452557    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:00.723564 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.723536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:32:00.725384 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:32:00.725359    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552cf52e_cd8b_4d1b_8e83_5ad73fe1c557.slice/crio-27bcf74555f90d31d8c3c3bd19fde87dd871cf451c1637872a3bccf6d95b6363 WatchSource:0}: Error finding container 27bcf74555f90d31d8c3c3bd19fde87dd871cf451c1637872a3bccf6d95b6363: Status 404 returned error can't find the container with id 27bcf74555f90d31d8c3c3bd19fde87dd871cf451c1637872a3bccf6d95b6363
Sep 18 20:32:00.986939 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.986847    2564 generic.go:356] "Generic (PLEG): container finished" podID="0974e185-f273-43a1-9384-7c9382e2e973" containerID="74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9" exitCode=2
Sep 18 20:32:00.986939 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.986908    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerDied","Data":"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"}
Sep 18 20:32:00.988743 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.988719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerStarted","Data":"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"}
Sep 18 20:32:00.988852 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.988769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerStarted","Data":"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"}
Sep 18 20:32:00.988852 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.988789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerStarted","Data":"27bcf74555f90d31d8c3c3bd19fde87dd871cf451c1637872a3bccf6d95b6363"}
Sep 18 20:32:00.988953 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:00.988938    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:01.008892 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:01.008840    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podStartSLOduration=1.008824692 podStartE2EDuration="1.008824692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:01.007584265 +0000 UTC m=+693.356856658" watchObservedRunningTime="2026-09-18 20:32:01.008824692 +0000 UTC m=+693.358097121"
Sep 18 20:32:01.993890 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:01.993860    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:01.995242 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:01.995213    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:02.789438 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:02.789400    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:32:02.999285 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:02.999243    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:03.717316 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.717291    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:32:03.860406 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.860315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwdxr\" (UniqueName: \"kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr\") pod \"0974e185-f273-43a1-9384-7c9382e2e973\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") "
Sep 18 20:32:03.860406 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.860385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls\") pod \"0974e185-f273-43a1-9384-7c9382e2e973\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") "
Sep 18 20:32:03.860624 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.860469    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config\") pod \"0974e185-f273-43a1-9384-7c9382e2e973\" (UID: \"0974e185-f273-43a1-9384-7c9382e2e973\") "
Sep 18 20:32:03.860838 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.860811    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0974e185-f273-43a1-9384-7c9382e2e973" (UID: "0974e185-f273-43a1-9384-7c9382e2e973"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:03.862580 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.862557    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0974e185-f273-43a1-9384-7c9382e2e973" (UID: "0974e185-f273-43a1-9384-7c9382e2e973"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:03.862701 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.862617    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr" (OuterVolumeSpecName: "kube-api-access-gwdxr") pod "0974e185-f273-43a1-9384-7c9382e2e973" (UID: "0974e185-f273-43a1-9384-7c9382e2e973"). InnerVolumeSpecName "kube-api-access-gwdxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:03.961845 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.961810    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0974e185-f273-43a1-9384-7c9382e2e973-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:03.961845 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.961839    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwdxr\" (UniqueName: \"kubernetes.io/projected/0974e185-f273-43a1-9384-7c9382e2e973-kube-api-access-gwdxr\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:03.961845 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:03.961850    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0974e185-f273-43a1-9384-7c9382e2e973-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:04.005482 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.005452    2564 generic.go:356] "Generic (PLEG): container finished" podID="0974e185-f273-43a1-9384-7c9382e2e973" containerID="e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52" exitCode=0
Sep 18 20:32:04.005886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.005496    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"
Sep 18 20:32:04.005886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.005537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerDied","Data":"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"}
Sep 18 20:32:04.005886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.005599    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf" event={"ID":"0974e185-f273-43a1-9384-7c9382e2e973","Type":"ContainerDied","Data":"1f25e390bd0e0c5512f1f9e00c8ceb25d1d0b08b8d0ea647e626605d3b6228bd"}
Sep 18 20:32:04.005886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.005648    2564 scope.go:117] "RemoveContainer" containerID="74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"
Sep 18 20:32:04.014134 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.014118    2564 scope.go:117] "RemoveContainer" containerID="e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"
Sep 18 20:32:04.021666 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.021652    2564 scope.go:117] "RemoveContainer" containerID="74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"
Sep 18 20:32:04.021921 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:04.021902    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9\": container with ID starting with 74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9 not found: ID does not exist" containerID="74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"
Sep 18 20:32:04.021998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.021932    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9"} err="failed to get container status \"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9\": rpc error: code = NotFound desc = could not find container \"74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9\": container with ID starting with 74506d1610187a9458f59ed92c9691ccd4e689c3fccee038c084d9258b7003e9 not found: ID does not exist"
Sep 18 20:32:04.021998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.021956    2564 scope.go:117] "RemoveContainer" containerID="e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"
Sep 18 20:32:04.022236 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:04.022219    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52\": container with ID starting with e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52 not found: ID does not exist" containerID="e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"
Sep 18 20:32:04.022278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.022242    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52"} err="failed to get container status \"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52\": rpc error: code = NotFound desc = could not find container \"e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52\": container with ID starting with e96fcb2dc4bb79eb5e605208c5e66efdc72c988329582675853079ce52eeea52 not found: ID does not exist"
Sep 18 20:32:04.028770 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.028749    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:32:04.034617 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:04.034594    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ef587-predictor-6d4f85455-lpmjf"]
Sep 18 20:32:05.793458 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:05.793425    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0974e185-f273-43a1-9384-7c9382e2e973" path="/var/lib/kubelet/pods/0974e185-f273-43a1-9384-7c9382e2e973/volumes"
Sep 18 20:32:08.004470 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:08.004442    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:08.005017 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:08.004990    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:18.005136 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:18.005057    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:28.005146 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:28.005107    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:30.616184 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.616151    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:32:30.616639 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.616396    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container" containerID="cri-o://4164f82afceffa2901ba02a10c652051f0694e5c47dff201e184313cbd02cdd5" gracePeriod=30
Sep 18 20:32:30.616639 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.616421    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kube-rbac-proxy" containerID="cri-o://4a031f462ebf52d51f4f253f8f0e6dd36437429bac5c611de2d11d0fd96744e0" gracePeriod=30
Sep 18 20:32:30.659900 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.659866    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:32:30.660110 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.660090    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container" containerID="cri-o://cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733" gracePeriod=30
Sep 18 20:32:30.660163 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.660139    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kube-rbac-proxy" containerID="cri-o://1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2" gracePeriod=30
Sep 18 20:32:30.700267 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.700237    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:32:30.700482 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.700454    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" containerID="cri-o://7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4" gracePeriod=30
Sep 18 20:32:30.700564 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.700488    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kube-rbac-proxy" containerID="cri-o://6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809" gracePeriod=30
Sep 18 20:32:30.800108 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800076    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:32:30.800451 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800439    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kube-rbac-proxy"
Sep 18 20:32:30.800494 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kube-rbac-proxy"
Sep 18 20:32:30.800494 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800472    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container"
Sep 18 20:32:30.800494 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800477    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container"
Sep 18 20:32:30.800581 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800530    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kserve-container"
Sep 18 20:32:30.800581 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.800540    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0974e185-f273-43a1-9384-7c9382e2e973" containerName="kube-rbac-proxy"
Sep 18 20:32:30.803538 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.803522    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.806937 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.806872    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2bced-kube-rbac-proxy-sar-config\""
Sep 18 20:32:30.806937 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.806879    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2bced-predictor-serving-cert\""
Sep 18 20:32:30.815577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.815553    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:32:30.895738 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.895650    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmdh6\" (UniqueName: \"kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.895738 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.895706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.895961 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.895790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.996804 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.996761    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.997079 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.996835    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmdh6\" (UniqueName: \"kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.997079 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.996883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.997486 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.997466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:30.999334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:30.999311    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:31.004956 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.004932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmdh6\" (UniqueName: \"kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6\") pod \"success-200-isvc-2bced-predictor-6885566469-55wjx\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") " pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:31.116943 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.116898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:31.136094 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.136059    2564 generic.go:356] "Generic (PLEG): container finished" podID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerID="4a031f462ebf52d51f4f253f8f0e6dd36437429bac5c611de2d11d0fd96744e0" exitCode=2
Sep 18 20:32:31.136462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.136138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerDied","Data":"4a031f462ebf52d51f4f253f8f0e6dd36437429bac5c611de2d11d0fd96744e0"}
Sep 18 20:32:31.138267 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.138246    2564 generic.go:356] "Generic (PLEG): container finished" podID="969b90fa-d180-44b9-b478-29eac744e4c0" containerID="1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2" exitCode=2
Sep 18 20:32:31.138372 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.138318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerDied","Data":"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"}
Sep 18 20:32:31.141188 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.140979    2564 generic.go:356] "Generic (PLEG): container finished" podID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerID="6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809" exitCode=2
Sep 18 20:32:31.141188 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.141016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerDied","Data":"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"}
Sep 18 20:32:31.392196 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:31.392167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:32:31.393308 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:32:31.393284    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a663e91_3b82_43be_bd95_a7f35ca31c33.slice/crio-e328c2e37b89bb97edf54fa8f9d6ab4e2fdc3cd820ca818c98cc9f337fa0ee11 WatchSource:0}: Error finding container e328c2e37b89bb97edf54fa8f9d6ab4e2fdc3cd820ca818c98cc9f337fa0ee11: Status 404 returned error can't find the container with id e328c2e37b89bb97edf54fa8f9d6ab4e2fdc3cd820ca818c98cc9f337fa0ee11
Sep 18 20:32:32.154739 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.154706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerStarted","Data":"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"}
Sep 18 20:32:32.155151 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.154772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerStarted","Data":"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"}
Sep 18 20:32:32.155151 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.154800    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:32.155151 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.154818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerStarted","Data":"e328c2e37b89bb97edf54fa8f9d6ab4e2fdc3cd820ca818c98cc9f337fa0ee11"}
Sep 18 20:32:32.174517 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.174450    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podStartSLOduration=2.174431912 podStartE2EDuration="2.174431912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:32.173117314 +0000 UTC m=+724.522389728" watchObservedRunningTime="2026-09-18 20:32:32.174431912 +0000 UTC m=+724.523704304"
Sep 18 20:32:32.789824 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:32.789776    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 18 20:32:33.160570 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:33.160482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:33.161980 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:33.161949    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:32:33.583847 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:33.583799    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Sep 18 20:32:33.584030 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:33.583800    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Sep 18 20:32:34.166231 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:34.166194    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:32:34.670991 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:34.670943    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Sep 18 20:32:35.151026 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.150993    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:32:35.173393 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.173357    2564 generic.go:356] "Generic (PLEG): container finished" podID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerID="7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4" exitCode=0
Sep 18 20:32:35.173808 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.173421    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"
Sep 18 20:32:35.173808 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.173438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerDied","Data":"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"}
Sep 18 20:32:35.173808 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.173503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66" event={"ID":"284290e5-8a67-4df3-a30d-34d3af70b7bc","Type":"ContainerDied","Data":"10bd41917aa7bef8a28e557a8ff4db02a5fad2504a7fe1aa858a982e0c72d6ea"}
Sep 18 20:32:35.173808 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.173532    2564 scope.go:117] "RemoveContainer" containerID="6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"
Sep 18 20:32:35.181855 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.181833    2564 scope.go:117] "RemoveContainer" containerID="7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"
Sep 18 20:32:35.189990 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.189966    2564 scope.go:117] "RemoveContainer" containerID="115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"
Sep 18 20:32:35.197754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.197731    2564 scope.go:117] "RemoveContainer" containerID="6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"
Sep 18 20:32:35.198034 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:35.198017    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809\": container with ID starting with 6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809 not found: ID does not exist" containerID="6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"
Sep 18 20:32:35.198100 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.198044    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809"} err="failed to get container status \"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809\": rpc error: code = NotFound desc = could not find container \"6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809\": container with ID starting with 6b0e96b198129d4b858d8cd91c0b1fc282b40563cf58004a714ba580b72e9809 not found: ID does not exist"
Sep 18 20:32:35.198100 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.198061    2564 scope.go:117] "RemoveContainer" containerID="7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"
Sep 18 20:32:35.198313 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:35.198294    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4\": container with ID starting with 7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4 not found: ID does not exist" containerID="7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"
Sep 18 20:32:35.198353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.198322    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4"} err="failed to get container status \"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4\": rpc error: code = NotFound desc = could not find container \"7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4\": container with ID starting with 7a87566ebb1c6d66976652c8c1440a0dc63e39f6054f72e390e1e07754e53ff4 not found: ID does not exist"
Sep 18 20:32:35.198353 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.198342    2564 scope.go:117] "RemoveContainer" containerID="115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"
Sep 18 20:32:35.198567 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:35.198553    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db\": container with ID starting with 115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db not found: ID does not exist" containerID="115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"
Sep 18 20:32:35.198688 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.198569    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db"} err="failed to get container status \"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db\": rpc error: code = NotFound desc = could not find container \"115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db\": container with ID starting with 115e87d6cbebb7c850f4fad45c03f149ce425de32d624a5111260d07c593c8db not found: ID does not exist"
Sep 18 20:32:35.229133 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229048    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config\") pod \"284290e5-8a67-4df3-a30d-34d3af70b7bc\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") "
Sep 18 20:32:35.229133 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls\") pod \"284290e5-8a67-4df3-a30d-34d3af70b7bc\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") "
Sep 18 20:32:35.229304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229149    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfpmv\" (UniqueName: \"kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv\") pod \"284290e5-8a67-4df3-a30d-34d3af70b7bc\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") "
Sep 18 20:32:35.229304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location\") pod \"284290e5-8a67-4df3-a30d-34d3af70b7bc\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") "
Sep 18 20:32:35.229304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229232    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert\") pod \"284290e5-8a67-4df3-a30d-34d3af70b7bc\" (UID: \"284290e5-8a67-4df3-a30d-34d3af70b7bc\") "
Sep 18 20:32:35.229550 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229524    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "284290e5-8a67-4df3-a30d-34d3af70b7bc" (UID: "284290e5-8a67-4df3-a30d-34d3af70b7bc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:35.229599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "284290e5-8a67-4df3-a30d-34d3af70b7bc" (UID: "284290e5-8a67-4df3-a30d-34d3af70b7bc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:35.229599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.229590    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "284290e5-8a67-4df3-a30d-34d3af70b7bc" (UID: "284290e5-8a67-4df3-a30d-34d3af70b7bc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:35.231349 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.231324    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "284290e5-8a67-4df3-a30d-34d3af70b7bc" (UID: "284290e5-8a67-4df3-a30d-34d3af70b7bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:35.231448 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.231357    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv" (OuterVolumeSpecName: "kube-api-access-sfpmv") pod "284290e5-8a67-4df3-a30d-34d3af70b7bc" (UID: "284290e5-8a67-4df3-a30d-34d3af70b7bc"). InnerVolumeSpecName "kube-api-access-sfpmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:35.329942 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.329900    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/284290e5-8a67-4df3-a30d-34d3af70b7bc-kserve-provision-location\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:35.329942 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.329934    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-cabundle-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:35.329942 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.329947    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:35.330179 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.329960    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/284290e5-8a67-4df3-a30d-34d3af70b7bc-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:35.330179 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.329973    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfpmv\" (UniqueName: \"kubernetes.io/projected/284290e5-8a67-4df3-a30d-34d3af70b7bc-kube-api-access-sfpmv\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:35.496864 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.496828    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:32:35.501062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.501032    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xtp66"]
Sep 18 20:32:35.793972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:35.793942    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" path="/var/lib/kubelet/pods/284290e5-8a67-4df3-a30d-34d3af70b7bc/volumes"
Sep 18 20:32:36.182929 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.182897    2564 generic.go:356] "Generic (PLEG): container finished" podID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerID="4164f82afceffa2901ba02a10c652051f0694e5c47dff201e184313cbd02cdd5" exitCode=0
Sep 18 20:32:36.183261 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.182967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerDied","Data":"4164f82afceffa2901ba02a10c652051f0694e5c47dff201e184313cbd02cdd5"}
Sep 18 20:32:36.277606 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.277580    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:32:36.337962 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.337879    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location\") pod \"06fd09d8-0967-424a-9535-35a2ccbf896e\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") "
Sep 18 20:32:36.337962 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.337913    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r95r9\" (UniqueName: \"kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9\") pod \"06fd09d8-0967-424a-9535-35a2ccbf896e\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") "
Sep 18 20:32:36.337962 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.337949    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls\") pod \"06fd09d8-0967-424a-9535-35a2ccbf896e\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") "
Sep 18 20:32:36.338208 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.337981    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config\") pod \"06fd09d8-0967-424a-9535-35a2ccbf896e\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") "
Sep 18 20:32:36.338208 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.337997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert\") pod \"06fd09d8-0967-424a-9535-35a2ccbf896e\" (UID: \"06fd09d8-0967-424a-9535-35a2ccbf896e\") "
Sep 18 20:32:36.338310 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.338256    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "06fd09d8-0967-424a-9535-35a2ccbf896e" (UID: "06fd09d8-0967-424a-9535-35a2ccbf896e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:36.338370 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.338336    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "06fd09d8-0967-424a-9535-35a2ccbf896e" (UID: "06fd09d8-0967-424a-9535-35a2ccbf896e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:36.338464 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.338442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "06fd09d8-0967-424a-9535-35a2ccbf896e" (UID: "06fd09d8-0967-424a-9535-35a2ccbf896e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:36.340062 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.340040    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "06fd09d8-0967-424a-9535-35a2ccbf896e" (UID: "06fd09d8-0967-424a-9535-35a2ccbf896e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:36.340191 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.340160    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9" (OuterVolumeSpecName: "kube-api-access-r95r9") pod "06fd09d8-0967-424a-9535-35a2ccbf896e" (UID: "06fd09d8-0967-424a-9535-35a2ccbf896e"). InnerVolumeSpecName "kube-api-access-r95r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:36.439508 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.439469    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06fd09d8-0967-424a-9535-35a2ccbf896e-kserve-provision-location\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:36.439508 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.439499    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r95r9\" (UniqueName: \"kubernetes.io/projected/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-api-access-r95r9\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:36.439508 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.439509    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06fd09d8-0967-424a-9535-35a2ccbf896e-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:36.439508 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.439518    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:36.439860 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.439527    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06fd09d8-0967-424a-9535-35a2ccbf896e-cabundle-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:36.993265 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:36.993236    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:32:37.042639 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042591    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location\") pod \"969b90fa-d180-44b9-b478-29eac744e4c0\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") "
Sep 18 20:32:37.042825 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042741    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert\") pod \"969b90fa-d180-44b9-b478-29eac744e4c0\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") "
Sep 18 20:32:37.042825 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042777    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config\") pod \"969b90fa-d180-44b9-b478-29eac744e4c0\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") "
Sep 18 20:32:37.042825 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042814    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffttj\" (UniqueName: \"kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj\") pod \"969b90fa-d180-44b9-b478-29eac744e4c0\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") "
Sep 18 20:32:37.042993 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042868    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls\") pod \"969b90fa-d180-44b9-b478-29eac744e4c0\" (UID: \"969b90fa-d180-44b9-b478-29eac744e4c0\") "
Sep 18 20:32:37.042993 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.042898    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "969b90fa-d180-44b9-b478-29eac744e4c0" (UID: "969b90fa-d180-44b9-b478-29eac744e4c0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:37.043097 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.043087    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/969b90fa-d180-44b9-b478-29eac744e4c0-kserve-provision-location\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:37.043181 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.043152    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "969b90fa-d180-44b9-b478-29eac744e4c0" (UID: "969b90fa-d180-44b9-b478-29eac744e4c0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:37.043237 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.043164    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "969b90fa-d180-44b9-b478-29eac744e4c0" (UID: "969b90fa-d180-44b9-b478-29eac744e4c0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:37.044998 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.044967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "969b90fa-d180-44b9-b478-29eac744e4c0" (UID: "969b90fa-d180-44b9-b478-29eac744e4c0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:37.045109 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.045017    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj" (OuterVolumeSpecName: "kube-api-access-ffttj") pod "969b90fa-d180-44b9-b478-29eac744e4c0" (UID: "969b90fa-d180-44b9-b478-29eac744e4c0"). InnerVolumeSpecName "kube-api-access-ffttj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:37.144117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.144025    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-cabundle-cert\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:37.144117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.144058    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/969b90fa-d180-44b9-b478-29eac744e4c0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:37.144117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.144071    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffttj\" (UniqueName: \"kubernetes.io/projected/969b90fa-d180-44b9-b478-29eac744e4c0-kube-api-access-ffttj\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:37.144117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.144084    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/969b90fa-d180-44b9-b478-29eac744e4c0-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:32:37.189307 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.189272    2564 generic.go:356] "Generic (PLEG): container finished" podID="969b90fa-d180-44b9-b478-29eac744e4c0" containerID="cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733" exitCode=0
Sep 18 20:32:37.189744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.189330    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"
Sep 18 20:32:37.189744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.189358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerDied","Data":"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"}
Sep 18 20:32:37.189744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.189428    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn" event={"ID":"969b90fa-d180-44b9-b478-29eac744e4c0","Type":"ContainerDied","Data":"85bd09f5d0539b6d9eca5505915c95d23d0532f7affbf1f5ad4ce20936f7f8a1"}
Sep 18 20:32:37.189744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.189469    2564 scope.go:117] "RemoveContainer" containerID="1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"
Sep 18 20:32:37.191580 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.191555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q" event={"ID":"06fd09d8-0967-424a-9535-35a2ccbf896e","Type":"ContainerDied","Data":"34805c33cee634bcd889db2faac47475e890f7398d8a2a982b7cef2d8ad0e82c"}
Sep 18 20:32:37.191696 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.191579    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"
Sep 18 20:32:37.199309 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.199160    2564 scope.go:117] "RemoveContainer" containerID="cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"
Sep 18 20:32:37.207293 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.207276    2564 scope.go:117] "RemoveContainer" containerID="4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"
Sep 18 20:32:37.215003 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.214982    2564 scope.go:117] "RemoveContainer" containerID="1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"
Sep 18 20:32:37.215353 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:37.215334    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2\": container with ID starting with 1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2 not found: ID does not exist" containerID="1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"
Sep 18 20:32:37.215428 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.215361    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2"} err="failed to get container status \"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2\": rpc error: code = NotFound desc = could not find container \"1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2\": container with ID starting with 1db719acc5991a2b260cb80632f25fb1c4febca7330b444df66f164ae5bdaee2 not found: ID does not exist"
Sep 18 20:32:37.215428 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.215384    2564 scope.go:117] "RemoveContainer" containerID="cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"
Sep 18 20:32:37.215709 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:37.215689    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733\": container with ID starting with cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733 not found: ID does not exist" containerID="cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"
Sep 18 20:32:37.215764 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.215713    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733"} err="failed to get container status \"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733\": rpc error: code = NotFound desc = could not find container \"cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733\": container with ID starting with cdd76b20b3c072b02f2c1b78b295497aee30c3a4ce542a915d7149b37f8e1733 not found: ID does not exist"
Sep 18 20:32:37.215764 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.215730    2564 scope.go:117] "RemoveContainer" containerID="4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"
Sep 18 20:32:37.215992 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:32:37.215975    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d\": container with ID starting with 4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d not found: ID does not exist" containerID="4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"
Sep 18 20:32:37.216038 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.215995    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d"} err="failed to get container status \"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d\": rpc error: code = NotFound desc = could not find container \"4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d\": container with ID starting with 4bf5ac5757210373c31fcde20f769e25b448cb7bac2df20fd5fba7654cd8077d not found: ID does not exist"
Sep 18 20:32:37.216038 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.216008    2564 scope.go:117] "RemoveContainer" containerID="4a031f462ebf52d51f4f253f8f0e6dd36437429bac5c611de2d11d0fd96744e0"
Sep 18 20:32:37.216811 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.216791    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:32:37.220674 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.220653    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-66bf4f5cc4-j5ccn"]
Sep 18 20:32:37.225502 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.225437    2564 scope.go:117] "RemoveContainer" containerID="4164f82afceffa2901ba02a10c652051f0694e5c47dff201e184313cbd02cdd5"
Sep 18 20:32:37.233249 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.233216    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:32:37.235579 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.235558    2564 scope.go:117] "RemoveContainer" containerID="02703fba67f4ff35d315286f7d8d49628aeaf55e7ea388300ecda976a8367508"
Sep 18 20:32:37.237002 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.236978    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-b7ccc5f99-gl67q"]
Sep 18 20:32:37.793734 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.793694    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" path="/var/lib/kubelet/pods/06fd09d8-0967-424a-9535-35a2ccbf896e/volumes"
Sep 18 20:32:37.794289 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:37.794269    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" path="/var/lib/kubelet/pods/969b90fa-d180-44b9-b478-29eac744e4c0/volumes"
Sep 18 20:32:38.005724 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:38.005688    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:39.171719 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:39.171687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:32:39.172200 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:39.172173    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:32:48.005158 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:48.005117    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:32:49.172462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:49.172422    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:32:58.005682 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:58.005648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:32:59.172673 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:32:59.172615    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:33:09.172232 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:33:09.172192    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:33:19.172975 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:33:19.172938    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:33:29.173667 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:33:29.173612    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:35:32.406902 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:35:32.406874    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:35:32.409145 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:35:32.409126    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:40:32.433184 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:40:32.433157    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:40:32.436668 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:40:32.436648    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:41:25.537286 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.537202    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:41:25.537827 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.537481    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" containerID="cri-o://ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d" gracePeriod=30
Sep 18 20:41:25.537827 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.537511    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy" containerID="cri-o://84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c" gracePeriod=30
Sep 18 20:41:25.596797 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.596759    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:41:25.597287 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597260    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container"
Sep 18 20:41:25.597287 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597280    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597299    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597308    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597323    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="storage-initializer"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597332    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="storage-initializer"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597345    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597354    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597368    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="storage-initializer"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597376    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="storage-initializer"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597388    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597397    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597408    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597416    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597427 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597427    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597435    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597446    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="storage-initializer"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="storage-initializer"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597521    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597530    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="284290e5-8a67-4df3-a30d-34d3af70b7bc" containerName="kserve-container"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597544    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kserve-container"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597553    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kube-rbac-proxy"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597562    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06fd09d8-0967-424a-9535-35a2ccbf896e" containerName="kserve-container"
Sep 18 20:41:25.597894 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.597570    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="969b90fa-d180-44b9-b478-29eac744e4c0" containerName="kube-rbac-proxy"
Sep 18 20:41:25.601008 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.600988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.603845 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.603826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c092f-kube-rbac-proxy-sar-config\""
Sep 18 20:41:25.603952 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.603823    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c092f-predictor-serving-cert\""
Sep 18 20:41:25.617063 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.617036    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:41:25.692521 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.692488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcczn\" (UniqueName: \"kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.692716 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.692544    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.692716 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.692571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.793548 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.793457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcczn\" (UniqueName: \"kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.793548 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.793509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.793548 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.793535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.793841 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:41:25.793658    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-c092f-predictor-serving-cert: secret "success-200-isvc-c092f-predictor-serving-cert" not found
Sep 18 20:41:25.793841 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:41:25.793738    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls podName:7ba44c99-99da-45f6-8567-c31576f19f3c nodeName:}" failed. No retries permitted until 2026-09-18 20:41:26.293715424 +0000 UTC m=+1258.642987811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls") pod "success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" (UID: "7ba44c99-99da-45f6-8567-c31576f19f3c") : secret "success-200-isvc-c092f-predictor-serving-cert" not found
Sep 18 20:41:25.794295 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.794264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:25.803057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:25.803035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcczn\" (UniqueName: \"kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:26.298404 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.298372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:26.300866 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.300845    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") pod \"success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") " pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:26.372842 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.372808    2564 generic.go:356] "Generic (PLEG): container finished" podID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerID="84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c" exitCode=2
Sep 18 20:41:26.373007 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.372879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerDied","Data":"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"}
Sep 18 20:41:26.511486 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.511440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:26.786901 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:26.786864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:41:26.790457 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:41:26.790426    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba44c99_99da_45f6_8567_c31576f19f3c.slice/crio-6a13800559c7ba2e72dc02e94063ad0d7620d9837336f8ec34217ce13a4a8570 WatchSource:0}: Error finding container 6a13800559c7ba2e72dc02e94063ad0d7620d9837336f8ec34217ce13a4a8570: Status 404 returned error can't find the container with id 6a13800559c7ba2e72dc02e94063ad0d7620d9837336f8ec34217ce13a4a8570
Sep 18 20:41:27.379344 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.379300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerStarted","Data":"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"}
Sep 18 20:41:27.379537 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.379373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerStarted","Data":"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"}
Sep 18 20:41:27.379537 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.379412    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:27.379537 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.379442    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:27.379537 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.379468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerStarted","Data":"6a13800559c7ba2e72dc02e94063ad0d7620d9837336f8ec34217ce13a4a8570"}
Sep 18 20:41:27.380879 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.380849    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:41:27.399334 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.399290    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podStartSLOduration=2.399274622 podStartE2EDuration="2.399274622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:27.397689319 +0000 UTC m=+1259.746961728" watchObservedRunningTime="2026-09-18 20:41:27.399274622 +0000 UTC m=+1259.748547015"
Sep 18 20:41:27.999906 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:27.999849    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Sep 18 20:41:28.005286 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:28.005254    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 18 20:41:28.389854 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:28.389809    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:41:33.000133 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.000090    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Sep 18 20:41:33.178959 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.178936    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:41:33.258501 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.258409    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls\") pod \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") "
Sep 18 20:41:33.258501 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.258484    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config\") pod \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") "
Sep 18 20:41:33.258716 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.258505    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr2sf\" (UniqueName: \"kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf\") pod \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\" (UID: \"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557\") "
Sep 18 20:41:33.258973 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.258948    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" (UID: "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:41:33.260613 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.260583    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" (UID: "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:41:33.260744 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.260670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf" (OuterVolumeSpecName: "kube-api-access-pr2sf") pod "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" (UID: "552cf52e-cd8b-4d1b-8e83-5ad73fe1c557"). InnerVolumeSpecName "kube-api-access-pr2sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:41:33.359899 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.359861    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:41:33.359899 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.359887    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr2sf\" (UniqueName: \"kubernetes.io/projected/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-kube-api-access-pr2sf\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:41:33.359899 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.359904    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:41:33.393573 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.393541    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:41:33.394103 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.394073    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:41:33.409797 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.409765    2564 generic.go:356] "Generic (PLEG): container finished" podID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerID="ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d" exitCode=0
Sep 18 20:41:33.409932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.409818    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"
Sep 18 20:41:33.409932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.409846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerDied","Data":"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"}
Sep 18 20:41:33.409932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.409898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm" event={"ID":"552cf52e-cd8b-4d1b-8e83-5ad73fe1c557","Type":"ContainerDied","Data":"27bcf74555f90d31d8c3c3bd19fde87dd871cf451c1637872a3bccf6d95b6363"}
Sep 18 20:41:33.409932 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.409925    2564 scope.go:117] "RemoveContainer" containerID="84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"
Sep 18 20:41:33.418923 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.418903    2564 scope.go:117] "RemoveContainer" containerID="ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"
Sep 18 20:41:33.434252 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.434220    2564 scope.go:117] "RemoveContainer" containerID="84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"
Sep 18 20:41:33.435291 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:41:33.435249    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c\": container with ID starting with 84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c not found: ID does not exist" containerID="84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"
Sep 18 20:41:33.435399 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.435309    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c"} err="failed to get container status \"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c\": rpc error: code = NotFound desc = could not find container \"84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c\": container with ID starting with 84bf4518cc885aaa7c6c0cc4b92848c43cd525f3a73046b48004c47d97fdc99c not found: ID does not exist"
Sep 18 20:41:33.435399 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.435337    2564 scope.go:117] "RemoveContainer" containerID="ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"
Sep 18 20:41:33.435699 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:41:33.435683    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d\": container with ID starting with ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d not found: ID does not exist" containerID="ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"
Sep 18 20:41:33.435766 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.435707    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d"} err="failed to get container status \"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d\": rpc error: code = NotFound desc = could not find container \"ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d\": container with ID starting with ec6001ae8dab5df308fe502133dc6736d515d036a5da392e33921bef35af008d not found: ID does not exist"
Sep 18 20:41:33.436580 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.436559    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:41:33.438945 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.438922    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b4a7-predictor-6fd6897c9f-ddwqm"]
Sep 18 20:41:33.793260 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:33.793217    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" path="/var/lib/kubelet/pods/552cf52e-cd8b-4d1b-8e83-5ad73fe1c557/volumes"
Sep 18 20:41:43.394336 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:43.394295    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:41:53.394364 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:53.394325    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:41:56.304469 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.304430    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:41:56.304911 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.304702    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" containerID="cri-o://8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b" gracePeriod=30
Sep 18 20:41:56.304911 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.304753    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kube-rbac-proxy" containerID="cri-o://54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494" gracePeriod=30
Sep 18 20:41:56.424255 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424219    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:41:56.424573 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424561    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy"
Sep 18 20:41:56.424613 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424574    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy"
Sep 18 20:41:56.424613 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424586    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container"
Sep 18 20:41:56.424613 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424591    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container"
Sep 18 20:41:56.424733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424668    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kube-rbac-proxy"
Sep 18 20:41:56.424733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.424676    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="552cf52e-cd8b-4d1b-8e83-5ad73fe1c557" containerName="kserve-container"
Sep 18 20:41:56.427577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.427560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.430253 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.430231    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e7889-predictor-serving-cert\""
Sep 18 20:41:56.430686 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.430671    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e7889-kube-rbac-proxy-sar-config\""
Sep 18 20:41:56.438371 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.438348    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:41:56.515524 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.515492    2564 generic.go:356] "Generic (PLEG): container finished" podID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerID="54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494" exitCode=2
Sep 18 20:41:56.515700 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.515529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerDied","Data":"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"}
Sep 18 20:41:56.558768 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.558687    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.558768 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.558750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssfft\" (UniqueName: \"kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.558944 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.558789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.660220 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.660180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.660382 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.660239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssfft\" (UniqueName: \"kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.660382 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.660278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.660978 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.660951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.662661 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.662617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.668885 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.668863    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssfft\" (UniqueName: \"kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft\") pod \"success-200-isvc-e7889-predictor-785b95f47f-nvfq8\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") " pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:56.738650 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:56.738599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:57.012551 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.012407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:41:57.014776 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:41:57.014750    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93fa377_d5ab_45a3_9d04_136d8c6873ba.slice/crio-ba59b197194a744d230d93892940f2e339f5b5efa3b19dcf6a0d8eb11f00f903 WatchSource:0}: Error finding container ba59b197194a744d230d93892940f2e339f5b5efa3b19dcf6a0d8eb11f00f903: Status 404 returned error can't find the container with id ba59b197194a744d230d93892940f2e339f5b5efa3b19dcf6a0d8eb11f00f903
Sep 18 20:41:57.522976 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.522940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerStarted","Data":"0c0636a84172d3a3c257eaf92562a4a36b0fb9d4de0185b784d8020f49c63596"}
Sep 18 20:41:57.523450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.523025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerStarted","Data":"f3c400e4ff862524797a97f820d8f960062d6302bb511ea96d7189482516c7eb"}
Sep 18 20:41:57.523450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.523057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerStarted","Data":"ba59b197194a744d230d93892940f2e339f5b5efa3b19dcf6a0d8eb11f00f903"}
Sep 18 20:41:57.523450 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.523118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:57.544838 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:57.544771    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podStartSLOduration=1.5447548150000001 podStartE2EDuration="1.544754815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:57.54343919 +0000 UTC m=+1289.892711602" watchObservedRunningTime="2026-09-18 20:41:57.544754815 +0000 UTC m=+1289.894027208"
Sep 18 20:41:58.527888 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:58.527851    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:41:58.529199 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:58.529174    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:41:59.166986 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:59.166938    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Sep 18 20:41:59.172361 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:59.172332    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 18 20:41:59.532354 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:59.532312    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:41:59.952429 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:41:59.952403    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:42:00.089886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.089773    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls\") pod \"0a663e91-3b82-43be-bd95-a7f35ca31c33\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") "
Sep 18 20:42:00.089886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.089832    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config\") pod \"0a663e91-3b82-43be-bd95-a7f35ca31c33\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") "
Sep 18 20:42:00.089886 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.089865    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmdh6\" (UniqueName: \"kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6\") pod \"0a663e91-3b82-43be-bd95-a7f35ca31c33\" (UID: \"0a663e91-3b82-43be-bd95-a7f35ca31c33\") "
Sep 18 20:42:00.090236 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.090211    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0a663e91-3b82-43be-bd95-a7f35ca31c33" (UID: "0a663e91-3b82-43be-bd95-a7f35ca31c33"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:42:00.091959 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.091932    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0a663e91-3b82-43be-bd95-a7f35ca31c33" (UID: "0a663e91-3b82-43be-bd95-a7f35ca31c33"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:00.092059 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.091969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6" (OuterVolumeSpecName: "kube-api-access-tmdh6") pod "0a663e91-3b82-43be-bd95-a7f35ca31c33" (UID: "0a663e91-3b82-43be-bd95-a7f35ca31c33"). InnerVolumeSpecName "kube-api-access-tmdh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:00.191395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.191351    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a663e91-3b82-43be-bd95-a7f35ca31c33-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.191395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.191388    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.191603 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.191406    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmdh6\" (UniqueName: \"kubernetes.io/projected/0a663e91-3b82-43be-bd95-a7f35ca31c33-kube-api-access-tmdh6\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.538118 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.538084    2564 generic.go:356] "Generic (PLEG): container finished" podID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerID="8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b" exitCode=0
Sep 18 20:42:00.538541 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.538133    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"
Sep 18 20:42:00.538541 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.538173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerDied","Data":"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"}
Sep 18 20:42:00.538541 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.538234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx" event={"ID":"0a663e91-3b82-43be-bd95-a7f35ca31c33","Type":"ContainerDied","Data":"e328c2e37b89bb97edf54fa8f9d6ab4e2fdc3cd820ca818c98cc9f337fa0ee11"}
Sep 18 20:42:00.538541 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.538263    2564 scope.go:117] "RemoveContainer" containerID="54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"
Sep 18 20:42:00.546663 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.546625    2564 scope.go:117] "RemoveContainer" containerID="8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"
Sep 18 20:42:00.554596 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.554573    2564 scope.go:117] "RemoveContainer" containerID="54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"
Sep 18 20:42:00.554866 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:00.554847    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494\": container with ID starting with 54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494 not found: ID does not exist" containerID="54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"
Sep 18 20:42:00.554930 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.554875    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494"} err="failed to get container status \"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494\": rpc error: code = NotFound desc = could not find container \"54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494\": container with ID starting with 54bd876018ab468f4c0db95f858b914d5c67242d902e348ee7f4547c163d2494 not found: ID does not exist"
Sep 18 20:42:00.554930 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.554891    2564 scope.go:117] "RemoveContainer" containerID="8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"
Sep 18 20:42:00.555149 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:00.555131    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b\": container with ID starting with 8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b not found: ID does not exist" containerID="8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"
Sep 18 20:42:00.555188 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.555155    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b"} err="failed to get container status \"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b\": rpc error: code = NotFound desc = could not find container \"8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b\": container with ID starting with 8bffe814bdf52c693e3f76d3a8c1ef5df79e5e0e488884da4ca0e2b692d2319b not found: ID does not exist"
Sep 18 20:42:00.562707 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.562679    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:42:00.566089 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:00.566068    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bced-predictor-6885566469-55wjx"]
Sep 18 20:42:01.793730 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:01.793690    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" path="/var/lib/kubelet/pods/0a663e91-3b82-43be-bd95-a7f35ca31c33/volumes"
Sep 18 20:42:03.395095 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:03.395051    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:42:04.536593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:04.536565    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:42:04.537074 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:04.537023    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:42:13.394422 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:13.394379    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 18 20:42:14.537878 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:14.537839    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:42:23.394768 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:23.394738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:42:24.537704 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:24.537664    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:42:34.537854 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:34.537817    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:42:44.537396 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:44.537317    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 18 20:42:46.215715 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.215679    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:42:46.216131 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.215908    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container" containerID="cri-o://8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0" gracePeriod=30
Sep 18 20:42:46.216131 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.215964    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kube-rbac-proxy" containerID="cri-o://5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1" gracePeriod=30
Sep 18 20:42:46.336750 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.336715    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:42:46.337329 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337311    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container"
Sep 18 20:42:46.337392 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337330    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container"
Sep 18 20:42:46.337392 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337342    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kube-rbac-proxy"
Sep 18 20:42:46.337392 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337350    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kube-rbac-proxy"
Sep 18 20:42:46.337505 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337462    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kserve-container"
Sep 18 20:42:46.337505 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.337475    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a663e91-3b82-43be-bd95-a7f35ca31c33" containerName="kube-rbac-proxy"
Sep 18 20:42:46.340984 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.340963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.343685 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.343660    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4d797-predictor-serving-cert\""
Sep 18 20:42:46.343800 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.343665    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4d797-kube-rbac-proxy-sar-config\""
Sep 18 20:42:46.351266 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.351242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:42:46.468475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.468393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmlwb\" (UniqueName: \"kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.468475 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.468435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.468715 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.468507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.569444 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.569414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmlwb\" (UniqueName: \"kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.569614 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.569460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.569704 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.569604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.569761 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:46.569721    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-4d797-predictor-serving-cert: secret "success-200-isvc-4d797-predictor-serving-cert" not found
Sep 18 20:42:46.569814 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:46.569781    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls podName:71ad26b6-5135-40b7-b11e-b9a88cfe0579 nodeName:}" failed. No retries permitted until 2026-09-18 20:42:47.069762209 +0000 UTC m=+1339.419034603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls") pod "success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" (UID: "71ad26b6-5135-40b7-b11e-b9a88cfe0579") : secret "success-200-isvc-4d797-predictor-serving-cert" not found
Sep 18 20:42:46.570338 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.570320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.579107 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.579079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmlwb\" (UniqueName: \"kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:46.746339 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.746257    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerID="5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1" exitCode=2
Sep 18 20:42:46.746480 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:46.746329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerDied","Data":"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"}
Sep 18 20:42:47.074843 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.074808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:47.077159 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.077129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") pod \"success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") " pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:47.252443 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.252396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:47.524326 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.524301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:42:47.525931 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:42:47.525904    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ad26b6_5135_40b7_b11e_b9a88cfe0579.slice/crio-585f3220db550b7abfcc517c31967d3996681bc9fcade7a76f027e0a78089b49 WatchSource:0}: Error finding container 585f3220db550b7abfcc517c31967d3996681bc9fcade7a76f027e0a78089b49: Status 404 returned error can't find the container with id 585f3220db550b7abfcc517c31967d3996681bc9fcade7a76f027e0a78089b49
Sep 18 20:42:47.753720 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.753679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerStarted","Data":"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"}
Sep 18 20:42:47.753906 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.753757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerStarted","Data":"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"}
Sep 18 20:42:47.753906 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.753788    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerStarted","Data":"585f3220db550b7abfcc517c31967d3996681bc9fcade7a76f027e0a78089b49"}
Sep 18 20:42:47.753906 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.753877    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:47.771438 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:47.771390    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podStartSLOduration=1.7713752010000001 podStartE2EDuration="1.771375201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:42:47.769970413 +0000 UTC m=+1340.119242823" watchObservedRunningTime="2026-09-18 20:42:47.771375201 +0000 UTC m=+1340.120647591"
Sep 18 20:42:48.387873 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:48.387832    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Sep 18 20:42:48.758538 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:48.758454    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:48.759907 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:48.759872    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:42:49.659482 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.659458    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:42:49.763872 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.763793    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerID="8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0" exitCode=0
Sep 18 20:42:49.763872 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.763845    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"
Sep 18 20:42:49.764057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.763881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerDied","Data":"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"}
Sep 18 20:42:49.764057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.763960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z" event={"ID":"7ba44c99-99da-45f6-8567-c31576f19f3c","Type":"ContainerDied","Data":"6a13800559c7ba2e72dc02e94063ad0d7620d9837336f8ec34217ce13a4a8570"}
Sep 18 20:42:49.764057 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.763989    2564 scope.go:117] "RemoveContainer" containerID="5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"
Sep 18 20:42:49.764232 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.764207    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:42:49.772007 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.771949    2564 scope.go:117] "RemoveContainer" containerID="8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"
Sep 18 20:42:49.778960 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.778945    2564 scope.go:117] "RemoveContainer" containerID="5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"
Sep 18 20:42:49.779201 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:49.779182    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1\": container with ID starting with 5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1 not found: ID does not exist" containerID="5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"
Sep 18 20:42:49.779263 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.779209    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1"} err="failed to get container status \"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1\": rpc error: code = NotFound desc = could not find container \"5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1\": container with ID starting with 5bc91af35d9288d57a0e8ad0d9213fc950897d97384e1c5b7e49af8005da4ff1 not found: ID does not exist"
Sep 18 20:42:49.779263 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.779227    2564 scope.go:117] "RemoveContainer" containerID="8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"
Sep 18 20:42:49.779484 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:42:49.779466    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0\": container with ID starting with 8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0 not found: ID does not exist" containerID="8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"
Sep 18 20:42:49.779530 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.779489    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0"} err="failed to get container status \"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0\": rpc error: code = NotFound desc = could not find container \"8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0\": container with ID starting with 8e05e35b09aca903cb59909fffed1970817b861c02ae07f16354622d72e007b0 not found: ID does not exist"
Sep 18 20:42:49.796145 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.796126    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") pod \"7ba44c99-99da-45f6-8567-c31576f19f3c\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") "
Sep 18 20:42:49.796238 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.796184    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config\") pod \"7ba44c99-99da-45f6-8567-c31576f19f3c\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") "
Sep 18 20:42:49.796238 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.796214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcczn\" (UniqueName: \"kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn\") pod \"7ba44c99-99da-45f6-8567-c31576f19f3c\" (UID: \"7ba44c99-99da-45f6-8567-c31576f19f3c\") "
Sep 18 20:42:49.796528 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.796508    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7ba44c99-99da-45f6-8567-c31576f19f3c" (UID: "7ba44c99-99da-45f6-8567-c31576f19f3c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:42:49.798112 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.798089    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn" (OuterVolumeSpecName: "kube-api-access-tcczn") pod "7ba44c99-99da-45f6-8567-c31576f19f3c" (UID: "7ba44c99-99da-45f6-8567-c31576f19f3c"). InnerVolumeSpecName "kube-api-access-tcczn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:49.798206 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.798125    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7ba44c99-99da-45f6-8567-c31576f19f3c" (UID: "7ba44c99-99da-45f6-8567-c31576f19f3c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:49.897355 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.897328    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:49.897355 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.897354    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcczn\" (UniqueName: \"kubernetes.io/projected/7ba44c99-99da-45f6-8567-c31576f19f3c-kube-api-access-tcczn\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:49.897575 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:49.897365    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ba44c99-99da-45f6-8567-c31576f19f3c-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:42:50.089994 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:50.089966    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:42:50.094160 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:50.094139    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c092f-predictor-75bf5bdb89-gtr4z"]
Sep 18 20:42:51.793874 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:51.793839    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" path="/var/lib/kubelet/pods/7ba44c99-99da-45f6-8567-c31576f19f3c/volumes"
Sep 18 20:42:54.537785 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:54.537753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:42:54.768837 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:54.768811    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:42:54.769263 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:42:54.769235    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:43:04.770279 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:04.770237    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:43:14.769745 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:14.769704    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:43:17.678538 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.678505    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:43:17.678916 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.678780    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container" containerID="cri-o://f3c400e4ff862524797a97f820d8f960062d6302bb511ea96d7189482516c7eb" gracePeriod=30
Sep 18 20:43:17.678916 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.678857    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kube-rbac-proxy" containerID="cri-o://0c0636a84172d3a3c257eaf92562a4a36b0fb9d4de0185b784d8020f49c63596" gracePeriod=30
Sep 18 20:43:17.750064 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750034    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:43:17.750390 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750375    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kube-rbac-proxy"
Sep 18 20:43:17.750390 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750394    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kube-rbac-proxy"
Sep 18 20:43:17.750487 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750423    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container"
Sep 18 20:43:17.750487 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750431    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container"
Sep 18 20:43:17.750554 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750519    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kube-rbac-proxy"
Sep 18 20:43:17.750554 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.750533    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba44c99-99da-45f6-8567-c31576f19f3c" containerName="kserve-container"
Sep 18 20:43:17.755550 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.755533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.758565 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.758542    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8cb8c-kube-rbac-proxy-sar-config\""
Sep 18 20:43:17.758681 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.758563    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8cb8c-predictor-serving-cert\""
Sep 18 20:43:17.764580 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.764547    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:43:17.821051 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.821015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.821185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.821062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.821185 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.821119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klcts\" (UniqueName: \"kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.889459 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.889426    2564 generic.go:356] "Generic (PLEG): container finished" podID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerID="0c0636a84172d3a3c257eaf92562a4a36b0fb9d4de0185b784d8020f49c63596" exitCode=2
Sep 18 20:43:17.889621 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.889499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerDied","Data":"0c0636a84172d3a3c257eaf92562a4a36b0fb9d4de0185b784d8020f49c63596"}
Sep 18 20:43:17.922163 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.922130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.922304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.922169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.922304 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.922192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klcts\" (UniqueName: \"kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.922787 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.922766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.924534 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.924513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:17.934725 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:17.934675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klcts\" (UniqueName: \"kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts\") pod \"success-200-isvc-8cb8c-predictor-765d78496f-kjmd6\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") " pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:18.066452 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.066418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:18.336226 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.336203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:43:18.338153 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:43:18.338130    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696d27c2_9d89_4457_a480_25d2cea389ee.slice/crio-1fb01040d834f7765f95370159c5d9ec24d3737079d14d1739bd04dac7079a30 WatchSource:0}: Error finding container 1fb01040d834f7765f95370159c5d9ec24d3737079d14d1739bd04dac7079a30: Status 404 returned error can't find the container with id 1fb01040d834f7765f95370159c5d9ec24d3737079d14d1739bd04dac7079a30
Sep 18 20:43:18.895540 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.895506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerStarted","Data":"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"}
Sep 18 20:43:18.896028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.895560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerStarted","Data":"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"}
Sep 18 20:43:18.896028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.895580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerStarted","Data":"1fb01040d834f7765f95370159c5d9ec24d3737079d14d1739bd04dac7079a30"}
Sep 18 20:43:18.896028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.895624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:18.896028 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.895684    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:18.897078 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.897052    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:43:18.914078 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:18.914034    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podStartSLOduration=1.914019785 podStartE2EDuration="1.914019785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:43:18.91326827 +0000 UTC m=+1371.262540704" watchObservedRunningTime="2026-09-18 20:43:18.914019785 +0000 UTC m=+1371.263292178"
Sep 18 20:43:19.533017 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:19.532974    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Sep 18 20:43:19.899856 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:19.899769    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:43:20.909200 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:20.909165    2564 generic.go:356] "Generic (PLEG): container finished" podID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerID="f3c400e4ff862524797a97f820d8f960062d6302bb511ea96d7189482516c7eb" exitCode=0
Sep 18 20:43:20.909518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:20.909238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerDied","Data":"f3c400e4ff862524797a97f820d8f960062d6302bb511ea96d7189482516c7eb"}
Sep 18 20:43:21.025973 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.025949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:43:21.047471 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.047444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config\") pod \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") "
Sep 18 20:43:21.047662 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.047497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls\") pod \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") "
Sep 18 20:43:21.047662 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.047526    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssfft\" (UniqueName: \"kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft\") pod \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\" (UID: \"a93fa377-d5ab-45a3-9d04-136d8c6873ba\") "
Sep 18 20:43:21.047891 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.047850    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a93fa377-d5ab-45a3-9d04-136d8c6873ba" (UID: "a93fa377-d5ab-45a3-9d04-136d8c6873ba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:43:21.049969 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.049945    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft" (OuterVolumeSpecName: "kube-api-access-ssfft") pod "a93fa377-d5ab-45a3-9d04-136d8c6873ba" (UID: "a93fa377-d5ab-45a3-9d04-136d8c6873ba"). InnerVolumeSpecName "kube-api-access-ssfft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:43:21.050084 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.050012    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a93fa377-d5ab-45a3-9d04-136d8c6873ba" (UID: "a93fa377-d5ab-45a3-9d04-136d8c6873ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:43:21.149210 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.149144    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssfft\" (UniqueName: \"kubernetes.io/projected/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-api-access-ssfft\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:43:21.149210 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.149184    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a93fa377-d5ab-45a3-9d04-136d8c6873ba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:43:21.149210 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.149202    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a93fa377-d5ab-45a3-9d04-136d8c6873ba-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:43:21.915101 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.915076    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"
Sep 18 20:43:21.915549 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.915097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8" event={"ID":"a93fa377-d5ab-45a3-9d04-136d8c6873ba","Type":"ContainerDied","Data":"ba59b197194a744d230d93892940f2e339f5b5efa3b19dcf6a0d8eb11f00f903"}
Sep 18 20:43:21.915549 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.915161    2564 scope.go:117] "RemoveContainer" containerID="0c0636a84172d3a3c257eaf92562a4a36b0fb9d4de0185b784d8020f49c63596"
Sep 18 20:43:21.923166 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.923148    2564 scope.go:117] "RemoveContainer" containerID="f3c400e4ff862524797a97f820d8f960062d6302bb511ea96d7189482516c7eb"
Sep 18 20:43:21.934316 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.934296    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:43:21.937883 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:21.937863    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e7889-predictor-785b95f47f-nvfq8"]
Sep 18 20:43:23.794035 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:23.793999    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" path="/var/lib/kubelet/pods/a93fa377-d5ab-45a3-9d04-136d8c6873ba/volumes"
Sep 18 20:43:24.769226 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:24.769189    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:43:24.904282 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:24.904257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:43:24.904659 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:24.904618    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:43:34.769864 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:34.769826    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 18 20:43:34.904976 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:34.904940    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:43:44.770577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:44.770547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:43:44.905597 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:44.905559    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:43:54.905715 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:43:54.905670    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:44:04.905018 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:44:04.904977    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:44:14.905582 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:44:14.905510    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:45:32.459651 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:45:32.459530    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:45:32.463723 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:45:32.463708    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:50:32.486048 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:50:32.485936    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:50:32.492787 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:50:32.491274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:52:11.577661 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.577606    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:52:11.578230 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.577902    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" containerID="cri-o://7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f" gracePeriod=30
Sep 18 20:52:11.578331 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.578298    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kube-rbac-proxy" containerID="cri-o://29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa" gracePeriod=30
Sep 18 20:52:11.663996 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.663945    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:52:11.664433 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664414    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container"
Sep 18 20:52:11.664518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664437    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container"
Sep 18 20:52:11.664518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664447    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kube-rbac-proxy"
Sep 18 20:52:11.664518 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664456    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kube-rbac-proxy"
Sep 18 20:52:11.664689 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664567    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kube-rbac-proxy"
Sep 18 20:52:11.664689 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.664584    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93fa377-d5ab-45a3-9d04-136d8c6873ba" containerName="kserve-container"
Sep 18 20:52:11.667984 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.667965    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.670948 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.670922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-854fa-predictor-serving-cert\""
Sep 18 20:52:11.671059 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.670927    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-854fa-kube-rbac-proxy-sar-config\""
Sep 18 20:52:11.676997 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.676453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:52:11.754742 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.754708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.754937 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.754768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.754937 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.754853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgsj\" (UniqueName: \"kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.855728 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.855621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgsj\" (UniqueName: \"kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.855728 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.855722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.856120 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.855792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.856120 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:11.855870    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-854fa-predictor-serving-cert: secret "success-200-isvc-854fa-predictor-serving-cert" not found
Sep 18 20:52:11.856120 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:11.855960    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls podName:02a37061-dcb5-4998-870d-81cc21136c9e nodeName:}" failed. No retries permitted until 2026-09-18 20:52:12.355942991 +0000 UTC m=+1904.705215363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls") pod "success-200-isvc-854fa-predictor-d9d5c496f-w5djj" (UID: "02a37061-dcb5-4998-870d-81cc21136c9e") : secret "success-200-isvc-854fa-predictor-serving-cert" not found
Sep 18 20:52:11.856459 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.856439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:11.866574 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:11.866548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgsj\" (UniqueName: \"kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:12.061852 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.061815    2564 generic.go:356] "Generic (PLEG): container finished" podID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerID="29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa" exitCode=2
Sep 18 20:52:12.062047 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.061855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerDied","Data":"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"}
Sep 18 20:52:12.359803 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.359768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:12.362207 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.362185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") pod \"success-200-isvc-854fa-predictor-d9d5c496f-w5djj\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") " pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:12.580607 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.580572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:12.850740 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:12.850704    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:52:12.853800 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:52:12.853771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a37061_dcb5_4998_870d_81cc21136c9e.slice/crio-32d8c181fd3550c1199ab21a3194c3038cee6559d4d29cc6c276c070325d4811 WatchSource:0}: Error finding container 32d8c181fd3550c1199ab21a3194c3038cee6559d4d29cc6c276c070325d4811: Status 404 returned error can't find the container with id 32d8c181fd3550c1199ab21a3194c3038cee6559d4d29cc6c276c070325d4811
Sep 18 20:52:13.069056 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:13.069025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerStarted","Data":"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"}
Sep 18 20:52:13.069229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:13.069080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerStarted","Data":"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"}
Sep 18 20:52:13.069229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:13.069102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerStarted","Data":"32d8c181fd3550c1199ab21a3194c3038cee6559d4d29cc6c276c070325d4811"}
Sep 18 20:52:13.069229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:13.069155    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:13.087589 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:13.087546    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podStartSLOduration=2.087523526 podStartE2EDuration="2.087523526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:52:13.085956056 +0000 UTC m=+1905.435228466" watchObservedRunningTime="2026-09-18 20:52:13.087523526 +0000 UTC m=+1905.436795919"
Sep 18 20:52:14.073672 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.073616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:14.074989 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.074963    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:14.829275 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.829252    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:52:14.882105 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.882083    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmlwb\" (UniqueName: \"kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb\") pod \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") "
Sep 18 20:52:14.882240 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.882140    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") pod \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") "
Sep 18 20:52:14.882240 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.882177    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config\") pod \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\" (UID: \"71ad26b6-5135-40b7-b11e-b9a88cfe0579\") "
Sep 18 20:52:14.882570 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.882546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "71ad26b6-5135-40b7-b11e-b9a88cfe0579" (UID: "71ad26b6-5135-40b7-b11e-b9a88cfe0579"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:52:14.884111 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.884077    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb" (OuterVolumeSpecName: "kube-api-access-pmlwb") pod "71ad26b6-5135-40b7-b11e-b9a88cfe0579" (UID: "71ad26b6-5135-40b7-b11e-b9a88cfe0579"). InnerVolumeSpecName "kube-api-access-pmlwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:52:14.884202 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.884122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "71ad26b6-5135-40b7-b11e-b9a88cfe0579" (UID: "71ad26b6-5135-40b7-b11e-b9a88cfe0579"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:52:14.982987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.982910    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmlwb\" (UniqueName: \"kubernetes.io/projected/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-api-access-pmlwb\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:14.982987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.982936    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71ad26b6-5135-40b7-b11e-b9a88cfe0579-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:14.982987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:14.982947    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71ad26b6-5135-40b7-b11e-b9a88cfe0579-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:15.080298 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080262    2564 generic.go:356] "Generic (PLEG): container finished" podID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerID="7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f" exitCode=0
Sep 18 20:52:15.080733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080320    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"
Sep 18 20:52:15.080733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerDied","Data":"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"}
Sep 18 20:52:15.080733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" event={"ID":"71ad26b6-5135-40b7-b11e-b9a88cfe0579","Type":"ContainerDied","Data":"585f3220db550b7abfcc517c31967d3996681bc9fcade7a76f027e0a78089b49"}
Sep 18 20:52:15.080733 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080465    2564 scope.go:117] "RemoveContainer" containerID="29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"
Sep 18 20:52:15.080940 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.080757    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:15.089462 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.089443    2564 scope.go:117] "RemoveContainer" containerID="7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"
Sep 18 20:52:15.096568 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.096553    2564 scope.go:117] "RemoveContainer" containerID="29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"
Sep 18 20:52:15.096861 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:15.096842    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa\": container with ID starting with 29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa not found: ID does not exist" containerID="29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"
Sep 18 20:52:15.096921 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.096869    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa"} err="failed to get container status \"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa\": rpc error: code = NotFound desc = could not find container \"29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa\": container with ID starting with 29c9e3fa513adafa13a31e4120dbaddf4adea51b89981cde42948c1d5057bfaa not found: ID does not exist"
Sep 18 20:52:15.096921 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.096885    2564 scope.go:117] "RemoveContainer" containerID="7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"
Sep 18 20:52:15.097143 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:15.097127    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f\": container with ID starting with 7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f not found: ID does not exist" containerID="7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"
Sep 18 20:52:15.097177 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.097149    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f"} err="failed to get container status \"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f\": rpc error: code = NotFound desc = could not find container \"7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f\": container with ID starting with 7d2f61bb7647821ea1fbf98f92e69c51da291875960282ff0beff2f2e24e9e3f not found: ID does not exist"
Sep 18 20:52:15.104207 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.104185    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:52:15.108322 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.108303    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2"]
Sep 18 20:52:15.765395 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.765350    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": context deadline exceeded"
Sep 18 20:52:15.770525 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.770498    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4d797-predictor-86bcd8f8c9-7bjt2" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: i/o timeout"
Sep 18 20:52:15.794082 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:15.794057    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" path="/var/lib/kubelet/pods/71ad26b6-5135-40b7-b11e-b9a88cfe0579/volumes"
Sep 18 20:52:20.085800 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:20.085771    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:52:20.086272 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:20.086185    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:30.086386 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:30.086347    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:40.086845 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:40.086808    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:43.059602 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.059563    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:52:43.060118 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.059821    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" containerID="cri-o://a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d" gracePeriod=30
Sep 18 20:52:43.060118 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.059907    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kube-rbac-proxy" containerID="cri-o://ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376" gracePeriod=30
Sep 18 20:52:43.196726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.196691    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 20:52:43.197050 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197030    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kube-rbac-proxy"
Sep 18 20:52:43.197050 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197049    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kube-rbac-proxy"
Sep 18 20:52:43.197229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197070    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container"
Sep 18 20:52:43.197229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197079    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container"
Sep 18 20:52:43.197229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197187    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kube-rbac-proxy"
Sep 18 20:52:43.197229 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.197209    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ad26b6-5135-40b7-b11e-b9a88cfe0579" containerName="kserve-container"
Sep 18 20:52:43.200435 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.200417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.203592 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.203567    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c7ffd-kube-rbac-proxy-sar-config\""
Sep 18 20:52:43.203715 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.203617    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c7ffd-predictor-serving-cert\""
Sep 18 20:52:43.212337 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.212314    2564 generic.go:356] "Generic (PLEG): container finished" podID="696d27c2-9d89-4457-a480-25d2cea389ee" containerID="ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376" exitCode=2
Sep 18 20:52:43.212437 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.212382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerDied","Data":"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"}
Sep 18 20:52:43.216206 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.216187    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 20:52:43.310555 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.310479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.310555 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.310518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.310778 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.310613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvfl\" (UniqueName: \"kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.412008 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.411974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvfl\" (UniqueName: \"kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.412168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.412021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.412168 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.412041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.412625 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.412604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.414577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.414556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.423300 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.423280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvfl\" (UniqueName: \"kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl\") pod \"success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") " pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.511073 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.511044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:43.782290 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:43.782265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 20:52:43.783817 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:52:43.783788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d14796a_8319_4501_adcf_c3fb32d62c20.slice/crio-bab70232e4f8587595a2ce101bfb889d4e24e8220d20dfb8e6bca87ef00389b2 WatchSource:0}: Error finding container bab70232e4f8587595a2ce101bfb889d4e24e8220d20dfb8e6bca87ef00389b2: Status 404 returned error can't find the container with id bab70232e4f8587595a2ce101bfb889d4e24e8220d20dfb8e6bca87ef00389b2
Sep 18 20:52:44.219282 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.219182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerStarted","Data":"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"}
Sep 18 20:52:44.219282 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.219245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerStarted","Data":"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"}
Sep 18 20:52:44.219282 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.219272    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:44.219726 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.219294    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerStarted","Data":"bab70232e4f8587595a2ce101bfb889d4e24e8220d20dfb8e6bca87ef00389b2"}
Sep 18 20:52:44.900151 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.900111    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Sep 18 20:52:44.905438 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:44.905412    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 18 20:52:45.223601 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:45.223518    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:45.224959 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:45.224933    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:52:46.227983 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.227939    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:52:46.506717 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.506692    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:52:46.528158 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.528109    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podStartSLOduration=3.528093196 podStartE2EDuration="3.528093196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:52:44.241480022 +0000 UTC m=+1936.590752438" watchObservedRunningTime="2026-09-18 20:52:46.528093196 +0000 UTC m=+1938.877365588"
Sep 18 20:52:46.538593 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.538566    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klcts\" (UniqueName: \"kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts\") pod \"696d27c2-9d89-4457-a480-25d2cea389ee\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") "
Sep 18 20:52:46.538730 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.538643    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls\") pod \"696d27c2-9d89-4457-a480-25d2cea389ee\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") "
Sep 18 20:52:46.538730 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.538691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config\") pod \"696d27c2-9d89-4457-a480-25d2cea389ee\" (UID: \"696d27c2-9d89-4457-a480-25d2cea389ee\") "
Sep 18 20:52:46.539137 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.539110    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "696d27c2-9d89-4457-a480-25d2cea389ee" (UID: "696d27c2-9d89-4457-a480-25d2cea389ee"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:52:46.540748 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.540718    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "696d27c2-9d89-4457-a480-25d2cea389ee" (UID: "696d27c2-9d89-4457-a480-25d2cea389ee"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:52:46.540972 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.540950    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts" (OuterVolumeSpecName: "kube-api-access-klcts") pod "696d27c2-9d89-4457-a480-25d2cea389ee" (UID: "696d27c2-9d89-4457-a480-25d2cea389ee"). InnerVolumeSpecName "kube-api-access-klcts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:52:46.639377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.639336    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/696d27c2-9d89-4457-a480-25d2cea389ee-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:46.639377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.639375    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/696d27c2-9d89-4457-a480-25d2cea389ee-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:46.639377 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:46.639385    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klcts\" (UniqueName: \"kubernetes.io/projected/696d27c2-9d89-4457-a480-25d2cea389ee-kube-api-access-klcts\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:52:47.233713 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.233680    2564 generic.go:356] "Generic (PLEG): container finished" podID="696d27c2-9d89-4457-a480-25d2cea389ee" containerID="a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d" exitCode=0
Sep 18 20:52:47.234085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.233730    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"
Sep 18 20:52:47.234085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.233740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerDied","Data":"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"}
Sep 18 20:52:47.234085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.233789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6" event={"ID":"696d27c2-9d89-4457-a480-25d2cea389ee","Type":"ContainerDied","Data":"1fb01040d834f7765f95370159c5d9ec24d3737079d14d1739bd04dac7079a30"}
Sep 18 20:52:47.234085 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.233816    2564 scope.go:117] "RemoveContainer" containerID="ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"
Sep 18 20:52:47.243333 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.243314    2564 scope.go:117] "RemoveContainer" containerID="a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"
Sep 18 20:52:47.251080 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.251061    2564 scope.go:117] "RemoveContainer" containerID="ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"
Sep 18 20:52:47.251336 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:47.251316    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376\": container with ID starting with ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376 not found: ID does not exist" containerID="ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"
Sep 18 20:52:47.251401 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.251343    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376"} err="failed to get container status \"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376\": rpc error: code = NotFound desc = could not find container \"ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376\": container with ID starting with ecaf040fbef796c6ec20b67c54255b08b0d010c60ea9d251d0305ef2fefb6376 not found: ID does not exist"
Sep 18 20:52:47.251401 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.251359    2564 scope.go:117] "RemoveContainer" containerID="a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"
Sep 18 20:52:47.251621 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:52:47.251602    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d\": container with ID starting with a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d not found: ID does not exist" containerID="a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"
Sep 18 20:52:47.251681 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.251646    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d"} err="failed to get container status \"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d\": rpc error: code = NotFound desc = could not find container \"a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d\": container with ID starting with a7e873e4ca9b0c6f7b9cc9466158a4f7b5ce6e3964fba27f1fe79f55af3b3e4d not found: ID does not exist"
Sep 18 20:52:47.260053 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.260029    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:52:47.264506 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.264482    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8cb8c-predictor-765d78496f-kjmd6"]
Sep 18 20:52:47.793276 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:47.793243    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" path="/var/lib/kubelet/pods/696d27c2-9d89-4457-a480-25d2cea389ee/volumes"
Sep 18 20:52:50.086837 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:50.086796    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:52:51.232719 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:51.232691    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:52:51.233108 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:52:51.233083    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:53:00.086280 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:00.086243    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 18 20:53:01.233754 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:01.233716    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:53:10.086866 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:10.086834    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:53:11.233567 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:11.233530    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:53:21.234153 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:21.234073    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:53:31.233402 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:31.233365    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Sep 18 20:53:32.708286 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.708254    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:53:32.708756 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.708454    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container" containerID="cri-o://a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043" gracePeriod=30
Sep 18 20:53:32.708756 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.708493    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kube-rbac-proxy" containerID="cri-o://846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8" gracePeriod=30
Sep 18 20:53:32.844549 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844517    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 20:53:32.844873 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844859    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kube-rbac-proxy"
Sep 18 20:53:32.844873 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844873    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kube-rbac-proxy"
Sep 18 20:53:32.844987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844888    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container"
Sep 18 20:53:32.844987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844894    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container"
Sep 18 20:53:32.844987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844973    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kube-rbac-proxy"
Sep 18 20:53:32.844987 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.844980    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="696d27c2-9d89-4457-a480-25d2cea389ee" containerName="kserve-container"
Sep 18 20:53:32.847914 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.847895    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:32.852284 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.852247    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8bda6-predictor-serving-cert\""
Sep 18 20:53:32.852573 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.852549    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8bda6-kube-rbac-proxy-sar-config\""
Sep 18 20:53:32.861999 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:32.861968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 20:53:33.005460 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.005369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.005460 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.005418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.005663 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.005528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9q72\" (UniqueName: \"kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.106877 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.106843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.107048 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.106912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9q72\" (UniqueName: \"kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.107048 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.106973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.107048 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:53:33.107037    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-serving-cert: secret "success-200-isvc-8bda6-predictor-serving-cert" not found
Sep 18 20:53:33.107225 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:53:33.107114    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls podName:1fccc1e4-ff57-4a28-b6f0-a84182dd1689 nodeName:}" failed. No retries permitted until 2026-09-18 20:53:33.607093117 +0000 UTC m=+1985.956365505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls") pod "success-200-isvc-8bda6-predictor-67c7c6c79-bg622" (UID: "1fccc1e4-ff57-4a28-b6f0-a84182dd1689") : secret "success-200-isvc-8bda6-predictor-serving-cert" not found
Sep 18 20:53:33.107655 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.107612    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.116315 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.116292    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9q72\" (UniqueName: \"kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.438003 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.437972    2564 generic.go:356] "Generic (PLEG): container finished" podID="02a37061-dcb5-4998-870d-81cc21136c9e" containerID="846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8" exitCode=2
Sep 18 20:53:33.438163 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.438029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerDied","Data":"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"}
Sep 18 20:53:33.610554 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.610505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.613041 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.613019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") pod \"success-200-isvc-8bda6-predictor-67c7c6c79-bg622\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") " pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:33.757813 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:33.757736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:34.029394 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.029367    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 20:53:34.031908 ip-10-0-140-180 kubenswrapper[2564]: W0918 20:53:34.031868    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fccc1e4_ff57_4a28_b6f0_a84182dd1689.slice/crio-c5404b2df9f55ef38dc27d8559f3583536f1cf2f097d1a8cd30b8bc9ddbbcad9 WatchSource:0}: Error finding container c5404b2df9f55ef38dc27d8559f3583536f1cf2f097d1a8cd30b8bc9ddbbcad9: Status 404 returned error can't find the container with id c5404b2df9f55ef38dc27d8559f3583536f1cf2f097d1a8cd30b8bc9ddbbcad9
Sep 18 20:53:34.446117 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.446012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerStarted","Data":"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"}
Sep 18 20:53:34.446289 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.446145    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:34.446289 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.446188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerStarted","Data":"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"}
Sep 18 20:53:34.446289 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.446225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerStarted","Data":"c5404b2df9f55ef38dc27d8559f3583536f1cf2f097d1a8cd30b8bc9ddbbcad9"}
Sep 18 20:53:34.446289 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.446257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:34.447452 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.447420    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:53:34.464807 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:34.464761    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podStartSLOduration=2.464749087 podStartE2EDuration="2.464749087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:53:34.463237231 +0000 UTC m=+1986.812509666" watchObservedRunningTime="2026-09-18 20:53:34.464749087 +0000 UTC m=+1986.814021480"
Sep 18 20:53:35.081499 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:35.081460    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.39:8643/healthz\": dial tcp 10.134.0.39:8643: connect: connection refused"
Sep 18 20:53:35.450577 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:35.450491    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:53:36.155211 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.155182    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:53:36.231473 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.231407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config\") pod \"02a37061-dcb5-4998-870d-81cc21136c9e\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") "
Sep 18 20:53:36.231599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.231479    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") pod \"02a37061-dcb5-4998-870d-81cc21136c9e\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") "
Sep 18 20:53:36.231599 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.231558    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slgsj\" (UniqueName: \"kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj\") pod \"02a37061-dcb5-4998-870d-81cc21136c9e\" (UID: \"02a37061-dcb5-4998-870d-81cc21136c9e\") "
Sep 18 20:53:36.231795 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.231770    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "02a37061-dcb5-4998-870d-81cc21136c9e" (UID: "02a37061-dcb5-4998-870d-81cc21136c9e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:53:36.233481 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.233458    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "02a37061-dcb5-4998-870d-81cc21136c9e" (UID: "02a37061-dcb5-4998-870d-81cc21136c9e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:53:36.233891 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.233871    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj" (OuterVolumeSpecName: "kube-api-access-slgsj") pod "02a37061-dcb5-4998-870d-81cc21136c9e" (UID: "02a37061-dcb5-4998-870d-81cc21136c9e"). InnerVolumeSpecName "kube-api-access-slgsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:53:36.332140 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.332113    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02a37061-dcb5-4998-870d-81cc21136c9e-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.332140 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.332137    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slgsj\" (UniqueName: \"kubernetes.io/projected/02a37061-dcb5-4998-870d-81cc21136c9e-kube-api-access-slgsj\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.332305 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.332147    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02a37061-dcb5-4998-870d-81cc21136c9e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.456112 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.456078    2564 generic.go:356] "Generic (PLEG): container finished" podID="02a37061-dcb5-4998-870d-81cc21136c9e" containerID="a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043" exitCode=0
Sep 18 20:53:36.456278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.456124    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"
Sep 18 20:53:36.456278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.456156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerDied","Data":"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"}
Sep 18 20:53:36.456278 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.456251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj" event={"ID":"02a37061-dcb5-4998-870d-81cc21136c9e","Type":"ContainerDied","Data":"32d8c181fd3550c1199ab21a3194c3038cee6559d4d29cc6c276c070325d4811"}
Sep 18 20:53:36.456442 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.456290    2564 scope.go:117] "RemoveContainer" containerID="846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"
Sep 18 20:53:36.464913 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.464898    2564 scope.go:117] "RemoveContainer" containerID="a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"
Sep 18 20:53:36.472004 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.471983    2564 scope.go:117] "RemoveContainer" containerID="846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"
Sep 18 20:53:36.472270 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:53:36.472251    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8\": container with ID starting with 846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8 not found: ID does not exist" containerID="846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"
Sep 18 20:53:36.472329 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.472277    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8"} err="failed to get container status \"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8\": rpc error: code = NotFound desc = could not find container \"846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8\": container with ID starting with 846ce7ae05a362716a25dc4b3dca721589c26b0702f2c24ac121ee09129f02d8 not found: ID does not exist"
Sep 18 20:53:36.472329 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.472293    2564 scope.go:117] "RemoveContainer" containerID="a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"
Sep 18 20:53:36.472540 ip-10-0-140-180 kubenswrapper[2564]: E0918 20:53:36.472526    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043\": container with ID starting with a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043 not found: ID does not exist" containerID="a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"
Sep 18 20:53:36.472579 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.472545    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043"} err="failed to get container status \"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043\": rpc error: code = NotFound desc = could not find container \"a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043\": container with ID starting with a5a1f2652243e1dc273102c6182ed5c427b4bec4055e92e34b2624c1ba1d5043 not found: ID does not exist"
Sep 18 20:53:36.478172 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.478151    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:53:36.480922 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:36.480901    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-854fa-predictor-d9d5c496f-w5djj"]
Sep 18 20:53:37.793282 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:37.793238    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" path="/var/lib/kubelet/pods/02a37061-dcb5-4998-870d-81cc21136c9e/volumes"
Sep 18 20:53:40.454715 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:40.454677    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:53:40.455332 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:40.455305    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:53:41.233767 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:41.233742    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 20:53:50.455403 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:53:50.455362    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:54:00.456033 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:54:00.455994    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:54:10.455708 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:54:10.455664    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:54:20.455491 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:54:20.455455    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 20:54:30.455782 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:54:30.455748    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 20:55:32.512178 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:55:32.512044    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 20:55:32.519425 ip-10-0-140-180 kubenswrapper[2564]: I0918 20:55:32.519407    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:00:32.539142 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:00:32.539114    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:00:32.547312 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:00:32.547295    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:02:58.035745 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:02:58.035713    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 21:02:58.036203 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:02:58.035941    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" containerID="cri-o://ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a" gracePeriod=30
Sep 18 21:02:58.036203 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:02:58.036013    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kube-rbac-proxy" containerID="cri-o://926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf" gracePeriod=30
Sep 18 21:02:58.736052 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:02:58.736015    2564 generic.go:356] "Generic (PLEG): container finished" podID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerID="926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf" exitCode=2
Sep 18 21:02:58.736226 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:02:58.736083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerDied","Data":"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"}
Sep 18 21:03:00.451721 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:00.451686    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": dial tcp 10.134.0.41:8643: connect: connection refused"
Sep 18 21:03:00.456114 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:00.456084    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 18 21:03:01.471895 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.471869    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 21:03:01.514846 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.514764    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") pod \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") "
Sep 18 21:03:01.514846 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.514799    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config\") pod \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") "
Sep 18 21:03:01.514846 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.514821    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9q72\" (UniqueName: \"kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72\") pod \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\" (UID: \"1fccc1e4-ff57-4a28-b6f0-a84182dd1689\") "
Sep 18 21:03:01.515126 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.515102    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1fccc1e4-ff57-4a28-b6f0-a84182dd1689" (UID: "1fccc1e4-ff57-4a28-b6f0-a84182dd1689"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 21:03:01.517000 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.516977    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72" (OuterVolumeSpecName: "kube-api-access-d9q72") pod "1fccc1e4-ff57-4a28-b6f0-a84182dd1689" (UID: "1fccc1e4-ff57-4a28-b6f0-a84182dd1689"). InnerVolumeSpecName "kube-api-access-d9q72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 21:03:01.517071 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.516998    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1fccc1e4-ff57-4a28-b6f0-a84182dd1689" (UID: "1fccc1e4-ff57-4a28-b6f0-a84182dd1689"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 21:03:01.616189 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.616153    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.616189 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.616184    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.616189 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.616193    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9q72\" (UniqueName: \"kubernetes.io/projected/1fccc1e4-ff57-4a28-b6f0-a84182dd1689-kube-api-access-d9q72\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.749849 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.749816    2564 generic.go:356] "Generic (PLEG): container finished" podID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerID="ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a" exitCode=0
Sep 18 21:03:01.749849 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.749854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerDied","Data":"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"}
Sep 18 21:03:01.750066 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.749869    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"
Sep 18 21:03:01.750066 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.749900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622" event={"ID":"1fccc1e4-ff57-4a28-b6f0-a84182dd1689","Type":"ContainerDied","Data":"c5404b2df9f55ef38dc27d8559f3583536f1cf2f097d1a8cd30b8bc9ddbbcad9"}
Sep 18 21:03:01.750066 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.749926    2564 scope.go:117] "RemoveContainer" containerID="926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"
Sep 18 21:03:01.758741 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.758723    2564 scope.go:117] "RemoveContainer" containerID="ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"
Sep 18 21:03:01.766053 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.766037    2564 scope.go:117] "RemoveContainer" containerID="926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"
Sep 18 21:03:01.766311 ip-10-0-140-180 kubenswrapper[2564]: E0918 21:03:01.766290    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf\": container with ID starting with 926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf not found: ID does not exist" containerID="926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"
Sep 18 21:03:01.766375 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.766319    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf"} err="failed to get container status \"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf\": rpc error: code = NotFound desc = could not find container \"926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf\": container with ID starting with 926314c3f7bebcc6633d8e99dd32c35d4ebf9bea7f98ff2b9f0a9b873d6dfbdf not found: ID does not exist"
Sep 18 21:03:01.766375 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.766335    2564 scope.go:117] "RemoveContainer" containerID="ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"
Sep 18 21:03:01.766580 ip-10-0-140-180 kubenswrapper[2564]: E0918 21:03:01.766564    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a\": container with ID starting with ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a not found: ID does not exist" containerID="ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"
Sep 18 21:03:01.766621 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.766584    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a"} err="failed to get container status \"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a\": rpc error: code = NotFound desc = could not find container \"ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a\": container with ID starting with ecc58fdad1293d0fa0edc9205af17a0f5f7cae394d28150a2b0d18e9d8548f5a not found: ID does not exist"
Sep 18 21:03:01.772128 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.772104    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 21:03:01.775300 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.775280    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8bda6-predictor-67c7c6c79-bg622"]
Sep 18 21:03:01.800831 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:03:01.800802    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" path="/var/lib/kubelet/pods/1fccc1e4-ff57-4a28-b6f0-a84182dd1689/volumes"
Sep 18 21:05:32.565373 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:05:32.565270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:05:32.574617 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:05:32.574599    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:10:13.650199 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:13.650157    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 21:10:13.650815 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:13.650396    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container" containerID="cri-o://e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b" gracePeriod=30
Sep 18 21:10:13.650815 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:13.650411    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kube-rbac-proxy" containerID="cri-o://bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c" gracePeriod=30
Sep 18 21:10:14.499937 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:14.499892    2564 generic.go:356] "Generic (PLEG): container finished" podID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerID="bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c" exitCode=2
Sep 18 21:10:14.500114 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:14.499964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerDied","Data":"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"}
Sep 18 21:10:16.228377 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:16.228331    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.40:8643/healthz\": dial tcp 10.134.0.40:8643: connect: connection refused"
Sep 18 21:10:16.992361 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:16.992338    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 21:10:17.094032 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.094003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config\") pod \"6d14796a-8319-4501-adcf-c3fb32d62c20\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") "
Sep 18 21:10:17.094172 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.094058    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvvfl\" (UniqueName: \"kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl\") pod \"6d14796a-8319-4501-adcf-c3fb32d62c20\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") "
Sep 18 21:10:17.094172 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.094130    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls\") pod \"6d14796a-8319-4501-adcf-c3fb32d62c20\" (UID: \"6d14796a-8319-4501-adcf-c3fb32d62c20\") "
Sep 18 21:10:17.094400 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.094381    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6d14796a-8319-4501-adcf-c3fb32d62c20" (UID: "6d14796a-8319-4501-adcf-c3fb32d62c20"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 21:10:17.096137 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.096077    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6d14796a-8319-4501-adcf-c3fb32d62c20" (UID: "6d14796a-8319-4501-adcf-c3fb32d62c20"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 21:10:17.096230 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.096129    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl" (OuterVolumeSpecName: "kube-api-access-bvvfl") pod "6d14796a-8319-4501-adcf-c3fb32d62c20" (UID: "6d14796a-8319-4501-adcf-c3fb32d62c20"). InnerVolumeSpecName "kube-api-access-bvvfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 21:10:17.195335 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.195302    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.195335 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.195329    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvvfl\" (UniqueName: \"kubernetes.io/projected/6d14796a-8319-4501-adcf-c3fb32d62c20-kube-api-access-bvvfl\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.195335 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.195339    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d14796a-8319-4501-adcf-c3fb32d62c20-proxy-tls\") on node \"ip-10-0-140-180.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.514954 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.514871    2564 generic.go:356] "Generic (PLEG): container finished" podID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerID="e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b" exitCode=0
Sep 18 21:10:17.514954 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.514934    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"
Sep 18 21:10:17.515449 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.514961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerDied","Data":"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"}
Sep 18 21:10:17.515449 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.515041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px" event={"ID":"6d14796a-8319-4501-adcf-c3fb32d62c20","Type":"ContainerDied","Data":"bab70232e4f8587595a2ce101bfb889d4e24e8220d20dfb8e6bca87ef00389b2"}
Sep 18 21:10:17.515449 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.515083    2564 scope.go:117] "RemoveContainer" containerID="bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"
Sep 18 21:10:17.523583 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.523569    2564 scope.go:117] "RemoveContainer" containerID="e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"
Sep 18 21:10:17.530560 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.530545    2564 scope.go:117] "RemoveContainer" containerID="bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"
Sep 18 21:10:17.530832 ip-10-0-140-180 kubenswrapper[2564]: E0918 21:10:17.530808    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c\": container with ID starting with bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c not found: ID does not exist" containerID="bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"
Sep 18 21:10:17.530909 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.530838    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c"} err="failed to get container status \"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c\": rpc error: code = NotFound desc = could not find container \"bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c\": container with ID starting with bce0a0e80d010e2a157cb79df8a5ecebaf4a6c3be8f3ba2e2a5a90ee29e7d46c not found: ID does not exist"
Sep 18 21:10:17.530909 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.530855    2564 scope.go:117] "RemoveContainer" containerID="e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"
Sep 18 21:10:17.531132 ip-10-0-140-180 kubenswrapper[2564]: E0918 21:10:17.531111    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b\": container with ID starting with e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b not found: ID does not exist" containerID="e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"
Sep 18 21:10:17.531189 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.531142    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b"} err="failed to get container status \"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b\": rpc error: code = NotFound desc = could not find container \"e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b\": container with ID starting with e791d9fe8d8a4a3f9f4d2fd85a64b23fc138d9c1fcdf13aaf6236b203c84bd6b not found: ID does not exist"
Sep 18 21:10:17.543331 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.543304    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 21:10:17.546373 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.546353    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c7ffd-predictor-7f499f4f6f-h99px"]
Sep 18 21:10:17.793372 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:17.793336    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" path="/var/lib/kubelet/pods/6d14796a-8319-4501-adcf-c3fb32d62c20/volumes"
Sep 18 21:10:32.596706 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:32.596578    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:10:32.606347 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:32.606327    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:10:46.275882 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.275842    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wnf42/must-gather-ngsv5"]
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276164    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276176    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276183    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276189    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276196    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276202    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276209    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276214    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276228    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276233    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276244    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276249    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276297    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276305    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276312    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kube-rbac-proxy"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276318    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d14796a-8319-4501-adcf-c3fb32d62c20" containerName="kserve-container"
Sep 18 21:10:46.276315 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276324    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fccc1e4-ff57-4a28-b6f0-a84182dd1689" containerName="kserve-container"
Sep 18 21:10:46.276937 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.276330    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a37061-dcb5-4998-870d-81cc21136c9e" containerName="kube-rbac-proxy"
Sep 18 21:10:46.279375 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.279349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.282310 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.282285    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wnf42\"/\"openshift-service-ca.crt\""
Sep 18 21:10:46.282418 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.282300    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wnf42\"/\"default-dockercfg-6nnkt\""
Sep 18 21:10:46.283558 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.283537    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wnf42\"/\"kube-root-ca.crt\""
Sep 18 21:10:46.288377 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.288354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/must-gather-ngsv5"]
Sep 18 21:10:46.333859 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.333817    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9580c639-5abe-43c5-9227-1c517989efee-must-gather-output\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.333859 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.333856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cv5g\" (UniqueName: \"kubernetes.io/projected/9580c639-5abe-43c5-9227-1c517989efee-kube-api-access-7cv5g\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.434699 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.434660    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9580c639-5abe-43c5-9227-1c517989efee-must-gather-output\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.434894 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.434705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cv5g\" (UniqueName: \"kubernetes.io/projected/9580c639-5abe-43c5-9227-1c517989efee-kube-api-access-7cv5g\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.435016 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.434992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9580c639-5abe-43c5-9227-1c517989efee-must-gather-output\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.443495 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.443466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cv5g\" (UniqueName: \"kubernetes.io/projected/9580c639-5abe-43c5-9227-1c517989efee-kube-api-access-7cv5g\") pod \"must-gather-ngsv5\" (UID: \"9580c639-5abe-43c5-9227-1c517989efee\") " pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.589261 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.589226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/must-gather-ngsv5"
Sep 18 21:10:46.859512 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.859486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/must-gather-ngsv5"]
Sep 18 21:10:46.861982 ip-10-0-140-180 kubenswrapper[2564]: W0918 21:10:46.861951    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9580c639_5abe_43c5_9227_1c517989efee.slice/crio-f620dfb922acdfc3f58af3f2d77953b9c8f906760231f32690428daac1603bae WatchSource:0}: Error finding container f620dfb922acdfc3f58af3f2d77953b9c8f906760231f32690428daac1603bae: Status 404 returned error can't find the container with id f620dfb922acdfc3f58af3f2d77953b9c8f906760231f32690428daac1603bae
Sep 18 21:10:46.864060 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:46.864045    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 21:10:47.654191 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:47.654166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/must-gather-ngsv5" event={"ID":"9580c639-5abe-43c5-9227-1c517989efee","Type":"ContainerStarted","Data":"2be1020d7c9228a6d2acc19bcaf73a6c920ce5f82acc5e1ba209bf068b76bae5"}
Sep 18 21:10:47.654470 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:47.654207    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/must-gather-ngsv5" event={"ID":"9580c639-5abe-43c5-9227-1c517989efee","Type":"ContainerStarted","Data":"f620dfb922acdfc3f58af3f2d77953b9c8f906760231f32690428daac1603bae"}
Sep 18 21:10:48.659788 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:48.659748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/must-gather-ngsv5" event={"ID":"9580c639-5abe-43c5-9227-1c517989efee","Type":"ContainerStarted","Data":"d4f3c87b12a2d96c0600088fbfa32d6354e1e6a970822fa9f3a3701078f4c4b3"}
Sep 18 21:10:48.690355 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:48.690292    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wnf42/must-gather-ngsv5" podStartSLOduration=1.993430671 podStartE2EDuration="2.690275501s" totalImagesPullingTime="696.84483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:10:48.687641908 +0000 UTC m=+3021.036914311" watchObservedRunningTime="2026-09-18 21:10:48.690275501 +0000 UTC m=+3021.039547894"
Sep 18 21:10:49.100320 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:49.100290    2564 ???:1] "http: TLS handshake error from 10.0.136.235:40946: EOF"
Sep 18 21:10:49.111252 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:49.111222    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7q8s6_392944fa-d16b-411b-86a8-62c6985ee1a5/global-pull-secret-syncer/0.log"
Sep 18 21:10:49.410654 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:49.410558    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tx7jf_4f45420e-b2a2-40d0-9180-d0a44bd128ef/konnectivity-agent/0.log"
Sep 18 21:10:49.495611 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:49.495579    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-180.ec2.internal_076cef5425bde9780a5fcdde6e01c168/haproxy/0.log"
Sep 18 21:10:53.223882 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.223853    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/alertmanager/0.log"
Sep 18 21:10:53.255992 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.255965    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/config-reloader/0.log"
Sep 18 21:10:53.318441 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.318409    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/kube-rbac-proxy-web/0.log"
Sep 18 21:10:53.393820 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.393790    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/kube-rbac-proxy/0.log"
Sep 18 21:10:53.451688 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.451654    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/kube-rbac-proxy-metric/0.log"
Sep 18 21:10:53.484414 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.484336    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/prom-label-proxy/0.log"
Sep 18 21:10:53.515303 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.515253    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f07853bf-849d-447c-9acb-f98e2c4f2121/init-config-reloader/0.log"
Sep 18 21:10:53.683873 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.683837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d8f95559c-qslpk_a08a2763-7271-48c5-a7e6-4e8170c3c45b/metrics-server/0.log"
Sep 18 21:10:53.939467 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.939436    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kmf8x_8f6a09a2-1ec0-41fb-8761-229f9f7e9be4/node-exporter/0.log"
Sep 18 21:10:53.968594 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.968570    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kmf8x_8f6a09a2-1ec0-41fb-8761-229f9f7e9be4/kube-rbac-proxy/0.log"
Sep 18 21:10:53.997964 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:53.997929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kmf8x_8f6a09a2-1ec0-41fb-8761-229f9f7e9be4/init-textfile/0.log"
Sep 18 21:10:54.034988 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.034945    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pwsvk_66f9638b-fc9a-495f-a586-c16ae1200516/kube-rbac-proxy-main/0.log"
Sep 18 21:10:54.073431 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.073374    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pwsvk_66f9638b-fc9a-495f-a586-c16ae1200516/kube-rbac-proxy-self/0.log"
Sep 18 21:10:54.103141 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.103111    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pwsvk_66f9638b-fc9a-495f-a586-c16ae1200516/openshift-state-metrics/0.log"
Sep 18 21:10:54.352425 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.352392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-42h26_71287f8c-e670-40a3-9cc9-ea3e21c09062/prometheus-operator/0.log"
Sep 18 21:10:54.387911 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.387862    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-42h26_71287f8c-e670-40a3-9cc9-ea3e21c09062/kube-rbac-proxy/0.log"
Sep 18 21:10:54.423428 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.423381    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-4p859_895bd099-6c92-498e-95f1-bea90c3b907e/prometheus-operator-admission-webhook/0.log"
Sep 18 21:10:54.476843 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.476812    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67b9dcc4cf-dc6sv_8d165f96-7aaf-4458-824e-b328e4f1f6c8/telemeter-client/0.log"
Sep 18 21:10:54.525392 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.525366    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67b9dcc4cf-dc6sv_8d165f96-7aaf-4458-824e-b328e4f1f6c8/reload/0.log"
Sep 18 21:10:54.563747 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:54.563705    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67b9dcc4cf-dc6sv_8d165f96-7aaf-4458-824e-b328e4f1f6c8/kube-rbac-proxy/0.log"
Sep 18 21:10:55.897360 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:55.897283    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-w558x_2c6f6f42-1937-4741-a7dd-5ba7b3710722/networking-console-plugin/0.log"
Sep 18 21:10:56.866177 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:56.866139    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85bf7d6cd4-v59zf_5e9a4603-c0b5-43f2-a118-e6ebd38287ab/console/0.log"
Sep 18 21:10:57.129557 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.129476    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"]
Sep 18 21:10:57.135446 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.135422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.144276 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.144247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"]
Sep 18 21:10:57.230516 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.230474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-lib-modules\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.230791 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.230770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-sys\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.230960 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.230939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-proc\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.231121 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.231105    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-podres\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.231293 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.231262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crwh\" (UniqueName: \"kubernetes.io/projected/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-kube-api-access-9crwh\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.332885 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.332842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-lib-modules\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.332919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-sys\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.332969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-proc\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.332997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-lib-modules\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.333027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-podres\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.333041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-sys\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.333058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-proc\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333073 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.333071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crwh\" (UniqueName: \"kubernetes.io/projected/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-kube-api-access-9crwh\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.333365 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.333150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-podres\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.342952 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.342918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crwh\" (UniqueName: \"kubernetes.io/projected/0bbc3283-a4ea-4059-ad6a-208dd32f3ec8-kube-api-access-9crwh\") pod \"perf-node-gather-daemonset-b79d4\" (UID: \"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.448732 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.448656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:57.740459 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:57.740401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"]
Sep 18 21:10:58.502816 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.502782    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mrj7p_bc8aa2d9-ec75-4b25-9d02-e31a55b59338/dns/0.log"
Sep 18 21:10:58.533438 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.533412    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mrj7p_bc8aa2d9-ec75-4b25-9d02-e31a55b59338/kube-rbac-proxy/0.log"
Sep 18 21:10:58.655104 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.655066    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8wqs_7e641a0c-60b0-40a9-bf65-7fea23ffdd29/dns-node-resolver/0.log"
Sep 18 21:10:58.723272 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.723224    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4" event={"ID":"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8","Type":"ContainerStarted","Data":"5f440957b689c4e47073d44aab88a60dc217f38de5266d490f52fc3bd854ba47"}
Sep 18 21:10:58.723454 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.723291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4" event={"ID":"0bbc3283-a4ea-4059-ad6a-208dd32f3ec8","Type":"ContainerStarted","Data":"178f1dca97279e0ee995d70620508428c37e4ba1ae0dc19241d6714161c28aa4"}
Sep 18 21:10:58.723454 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.723320    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:10:58.757493 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:58.755657    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4" podStartSLOduration=1.755618638 podStartE2EDuration="1.755618638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:10:58.744059264 +0000 UTC m=+3031.093331658" watchObservedRunningTime="2026-09-18 21:10:58.755618638 +0000 UTC m=+3031.104891032"
Sep 18 21:10:59.190534 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:10:59.190505    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2hj9w_66791213-4af8-4a60-9399-19e12eb2b352/node-ca/0.log"
Sep 18 21:11:00.428618 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:00.428590    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ddcx_cb5ab505-4689-4946-bb53-1e5c4fffe7a9/serve-healthcheck-canary/0.log"
Sep 18 21:11:01.029301 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:01.029270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqldl_9d8cb2d0-ec33-425c-a64c-8027c501a5ca/kube-rbac-proxy/0.log"
Sep 18 21:11:01.052515 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:01.052487    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqldl_9d8cb2d0-ec33-425c-a64c-8027c501a5ca/exporter/0.log"
Sep 18 21:11:01.080138 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:01.080116    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqldl_9d8cb2d0-ec33-425c-a64c-8027c501a5ca/extractor/0.log"
Sep 18 21:11:03.449686 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:03.449655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7dcf4bff6-zphhn_a2aedfb4-48bb-4b1d-b415-b25361564cbb/manager/0.log"
Sep 18 21:11:03.757743 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:03.757660    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-nmvrm_1dccb575-3dea-4a09-858b-5e64cc905e47/s3-init/0.log"
Sep 18 21:11:03.780341 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:03.780314    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-nmvrm_1dccb575-3dea-4a09-858b-5e64cc905e47/download-hf-model/0.log"
Sep 18 21:11:04.740016 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:04.739989    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-b79d4"
Sep 18 21:11:08.450235 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:08.450207    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-8st7g_c733f2b8-3449-45b5-91c3-5ea7db22ac77/migrator/0.log"
Sep 18 21:11:08.474682 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:08.474656    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-8st7g_c733f2b8-3449-45b5-91c3-5ea7db22ac77/graceful-termination/0.log"
Sep 18 21:11:09.811150 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.811124    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/kube-multus-additional-cni-plugins/0.log"
Sep 18 21:11:09.835108 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.835083    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/egress-router-binary-copy/0.log"
Sep 18 21:11:09.862174 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.862148    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/cni-plugins/0.log"
Sep 18 21:11:09.885522 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.885491    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/bond-cni-plugin/0.log"
Sep 18 21:11:09.911948 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.911920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/routeoverride-cni/0.log"
Sep 18 21:11:09.936651 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.936601    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/whereabouts-cni-bincopy/0.log"
Sep 18 21:11:09.961106 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:09.961081    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9hpx_8b96350d-9e01-4ec8-9ac7-3f4929a6eac4/whereabouts-cni/0.log"
Sep 18 21:11:10.480216 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:10.480181    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hpt84_236aafa2-de48-4bf3-b41a-707da2d61501/kube-multus/0.log"
Sep 18 21:11:10.686056 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:10.686018    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s4rsp_17ece2c5-d02f-4898-871f-313bb14f89ed/network-metrics-daemon/0.log"
Sep 18 21:11:10.708291 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:10.708261    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s4rsp_17ece2c5-d02f-4898-871f-313bb14f89ed/kube-rbac-proxy/0.log"
Sep 18 21:11:11.473903 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.473869    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-controller/0.log"
Sep 18 21:11:11.492195 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.492165    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/1.log"
Sep 18 21:11:11.492353 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.492287    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovn-acl-logging/0.log"
Sep 18 21:11:11.520546 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.520503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/kube-rbac-proxy-node/0.log"
Sep 18 21:11:11.544991 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.544953    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/kube-rbac-proxy-ovn-metrics/0.log"
Sep 18 21:11:11.566668 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.566638    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/northd/0.log"
Sep 18 21:11:11.591477 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.591446    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/nbdb/0.log"
Sep 18 21:11:11.614290 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.614266    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/sbdb/0.log"
Sep 18 21:11:11.673790 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:11.673752    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7nqtg_3976f7a8-ad32-42e8-ac8b-7b71abafe3c7/ovnkube-controller/0.log"
Sep 18 21:11:13.139243 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:13.139165    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-tgmm5_c25929c2-99dc-4114-be12-f46793185392/check-endpoints/0.log"
Sep 18 21:11:13.216291 ip-10-0-140-180 kubenswrapper[2564]: I0918 21:11:13.216256    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-x5lgz_8cabf748-848f-435c-af08-77e33aa4bd09/network-check-target-container/0.log"