Sep 11 16:17:16.087734 ip-10-0-128-17 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 16:17:16.512483 ip-10-0-128-17 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:16.512483 ip-10-0-128-17 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 16:17:16.512483 ip-10-0-128-17 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:16.512483 ip-10-0-128-17 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 16:17:16.512483 ip-10-0-128-17 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:16.513182 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.512608    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 16:17:16.518248 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518225    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.518248 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518245    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.518248 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518250    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.518248 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518254    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518259    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518264    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518269    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518275    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518279    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518284    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518288    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518293    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518297    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518301    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518305    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518318    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518323    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518327    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518331    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518335    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518339    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518343    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.518626 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518347    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518351    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518355    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518359    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518363    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518367    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518371    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518375    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518379    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518383    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518387    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518391    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518395    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518416    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518420    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518426    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518431    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518435    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518439    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518442    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.519433 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518447    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518451    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518455    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518459    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518463    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518466    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518471    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518475    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518479    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518483    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518487    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518492    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518496    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518500    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518504    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518508    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518513    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518520    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518527    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518532    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.519984 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518536    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518540    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518545    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518549    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518553    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518558    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518562    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518567    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518572    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518576    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518580    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518584    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518588    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518593    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518598    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518602    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518607    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518611    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518616    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.518627    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.520627 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519199    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519221    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519227    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519235    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519242    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519248    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519253    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519258    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519264    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519269    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519275    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519286    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519291    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519297    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519301    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519306    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.521121 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.519311    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521217    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521227    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521231    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521234    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521237    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521240    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521242    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521245    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521250    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521255    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521258    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521261    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521263    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521266    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521268    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521271    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521274    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521277    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.521547 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521280    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521283    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521286    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521288    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521291    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521294    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521297    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521300    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521304    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521308    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521312    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521315    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521318    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521321    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521324    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521327    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521330    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521333    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521336    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.522050 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521338    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521341    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521343    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521347    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521350    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521353    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521356    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521358    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521361    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521363    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521366    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521369    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521372    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521374    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521377    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521380    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521382    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521384    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521387    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521390    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.522554 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521393    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521396    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521424    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521428    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521430    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521433    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521436    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521438    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521441    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521444    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521447    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521449    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521452    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521454    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521457    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521460    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521462    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521465    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521467    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521470    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.523043 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521472    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521477    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521479    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521482    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521484    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521487    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521489    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521492    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521495    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521497    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521500    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521503    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521505    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521508    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521510    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521513    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521516    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521518    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521520    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521523    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.523552 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521527    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521530    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521900    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521905    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521907    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521910    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521912    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521915    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521918    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521921    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521924    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521926    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521929    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521932    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521935    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521939    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521943    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521946    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521949    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.524053 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521952    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521955    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521958    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521960    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521963    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521965    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521968    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521972    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521975    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521978    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521981    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521984    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521987    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521990    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521992    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521995    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.521998    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522002    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522005    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.524532 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522008    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522011    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522013    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522016    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522018    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522021    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522023    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522026    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522029    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522031    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522033    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522036    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522039    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522041    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522044    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522046    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522048    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522052    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522055    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522058    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.525009 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522060    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522063    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522066    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522068    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522071    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522074    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522076    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522079    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522081    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522084    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522087    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522089    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522092    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522094    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522096    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522100    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522102    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522105    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522107    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522109    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.525534 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522112    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522114    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522117    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522119    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522122    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522124    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522126    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522129    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522131    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522134    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522136    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522138    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522141    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522143    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522146    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522148    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522151    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522153    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522156    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522159    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.526028 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522161    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522163    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522167    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522237    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522272    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522279    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522283    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522288    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522291    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522295    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522299    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522302    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522305    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522309    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522312    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522315    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522318    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522321    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522324    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522327    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522329    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522333    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522336    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522339    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 16:17:16.526530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522342    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522345    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522348    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522351    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522354    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522358    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522361    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522364    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522366    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522369    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522372    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522376    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522379    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522382    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522385    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522388    2566 flags.go:64] FLAG: --enable-server="true"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522391    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522396    2566 flags.go:64] FLAG: --event-burst="100"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522414    2566 flags.go:64] FLAG: --event-qps="50"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522419    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522423    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522427    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522430    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522434    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522437    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 16:17:16.527108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522440    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522443    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522446    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522448    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522451    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522454    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522457    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522459    2566 flags.go:64] FLAG: --feature-gates=""
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522463    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522466    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522469    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522472    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522475    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522478    2566 flags.go:64] FLAG: --help="false"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522481    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522484    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522487    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522490    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522493    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522497    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522500    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522503    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522505    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 16:17:16.527765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522508    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522511    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522514    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522517    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522520    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522522    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522525    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522528    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522531    2566 flags.go:64] FLAG: --lock-file=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522533    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522536    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522539    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522544    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522547    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522550    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522553    2566 flags.go:64] FLAG: --logging-format="text"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522555    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522559    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522562    2566 flags.go:64] FLAG: --manifest-url=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522567    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522571    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522574    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522578    2566 flags.go:64] FLAG: --max-pods="110"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522581    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522584    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 16:17:16.528465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522587    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522590    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522593    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522595    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522600    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522607    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522610    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522613    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522616    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522619    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522625    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522627    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522630    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522633    2566 flags.go:64] FLAG: --port="10250"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522636    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522639    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c1fc190096fa2339"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522642    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522645    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522648    2566 flags.go:64] FLAG: --register-node="true"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522651    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522654    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522657    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522660    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522663    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522666    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 16:17:16.529186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522669    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522672    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522676    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522680    2566 flags.go:64] FLAG: --runonce="false"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522683    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522686    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522689    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522692    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522694    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522697    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522700    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522703    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522707    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522709    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522712    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522715    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522718    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522721    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522724    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522729    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522731    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522734    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522738    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522741    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522743    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 16:17:16.530163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522746    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522749    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522752    2566 flags.go:64] FLAG: --v="2"
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522756    2566 flags.go:64] FLAG: --version="false"
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522759    2566 flags.go:64] FLAG: --vmodule=""
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522763    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.522766    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522875    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522879    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522886    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522890    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522893    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522896    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522899    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522902    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522904    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522907    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522910    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522912    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522915    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522918    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.531283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522921    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522924    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522927    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522929    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522932    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522935    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522937    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522940    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522943    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522945    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522948    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522950    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522953    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522955    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522958    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522961    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522963    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522966    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522968    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522971    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.532245 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522976    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522979    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522982    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522984    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522987    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522989    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522992    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522995    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.522997    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523000    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523002    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523005    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523008    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523011    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523014    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523016    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523019    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523021    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523024    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523027    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.533188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523030    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523032    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523035    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523037    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523040    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523042    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523045    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523048    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523051    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523054    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523056    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523059    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523064    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523067    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523070    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523073    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523075    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523078    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523080    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523083    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.533885 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523086    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523090    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523092    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523096    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523100    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523107    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523110    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523113    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523116    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523119    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523122    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523125    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523127    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523130    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523133    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523136    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523139    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523142    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523144    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.534776 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523147    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523149    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523152    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523154    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523157    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.523163    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.523830    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.531655    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.531674    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.531700    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.531709    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531791    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531799    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531804    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.535358 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531810    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531815    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531820    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531825    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531829    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531835    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531839    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531844    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531849    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531854    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531858    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531863    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531867    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531871    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531876    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531880    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531884    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531888    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531893    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.535865 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531897    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531901    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531905    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531911    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531915    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531920    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531924    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531931    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531936    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531940    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531945    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531949    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531954    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531959    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531963    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531967    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531972    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531976    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531981    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531985    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.536339 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531990    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531994    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.531999    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532003    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532007    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532012    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532017    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532022    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532026    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532031    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532036    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532041    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532045    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532050    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532054    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532058    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532063    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532067    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532071    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532076    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.536912 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532082    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532086    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532091    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532097    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532104    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532110    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532114    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532119    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532123    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532130    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532136    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532141    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532146    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532150    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532154    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532158    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532163    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532167    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532172    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532176    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.537556 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532181    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532185    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532189    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532194    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532198    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532202    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532207    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532211    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532215    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532220    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532224    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532229    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532234    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532241    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532247    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532251    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.538102 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532256    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.532264    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532468    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532478    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532483    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532488    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532493    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532499    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532504    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532508    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532512    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532517    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532522    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532526    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532530    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:16.538515 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532535    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532540    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532544    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532549    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532553    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532557    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532561    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532566    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532570    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532575    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532580    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532584    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532589    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532594    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532598    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532603    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532608    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532613    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532617    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532621    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:16.538963 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532628    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532634    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532639    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532644    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532649    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532654    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532659    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532664    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532668    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532673    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532677    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532681    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532686    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532690    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532695    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532699    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532703    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532708    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532713    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532717    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:16.539469 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532724    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532728    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532733    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532738    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532743    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532748    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532752    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532756    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532762    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532766    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532770    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532775    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532779    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532784    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532788    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532793    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532797    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532801    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532806    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532810    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:16.540320 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532814    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532819    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532823    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532827    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532832    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532839    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532845    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532849    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532853    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532858    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532862    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532866    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532871    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532875    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532879    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532883    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532888    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532892    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532896    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532900    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:16.541256 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532905    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532910    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532915    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532919    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532924    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:16.532928    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.532937    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.533096    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.535538    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.536746    2566 server.go:1019] "Starting client certificate rotation"
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.536846    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.536877    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:17:16.541787 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.540097    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:17:16.543410 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.543376    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:17:16.554920 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.554903    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 11 16:17:16.558377 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.558363    2566 log.go:25] "Validated CRI v1 image API"
Sep 11 16:17:16.560156 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.560139    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 16:17:16.563194 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.563171    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 849794a8-1951-4b69-b988-f0d2aa2fd3cb:/dev/nvme0n1p4 eb471bb4-3319-4e02-96e0-4d1c22b46510:/dev/nvme0n1p3]
Sep 11 16:17:16.563264 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.563194    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 16:17:16.566160 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.566142    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:17:16.569612 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.569514    2566 manager.go:216] Machine: {Timestamp:2026-09-11 16:17:16.567999875 +0000 UTC m=+0.361041662 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098883 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2304fa1470925571da2a56cf676eec SystemUUID:ec2304fa-1470-9255-71da-2a56cf676eec BootID:1c934895-7e83-4569-ae28-0b4533220ed8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:ab:3d:e1:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:ab:3d:e1:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:cf:d9:8d:61:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 16:17:16.569612 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.569605    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 16:17:16.569722 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.569672    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 16:17:16.571358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.571339    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 16:17:16.571491 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.571361    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 16:17:16.572126 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.572116    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 16:17:16.572157 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.572129    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 16:17:16.572157 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.572136    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:17:16.572157 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.572150    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:17:16.572911 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.572901    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 16:17:16.576733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576722    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:17:16.576847 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576838    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 16:17:16.576877 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576866    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 16:17:16.576905 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576877    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 16:17:16.576905 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576888    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 16:17:16.576905 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576897    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 11 16:17:16.576905 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.576904    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 16:17:16.577205 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.577194    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:17:16.577249 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.577212    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:17:16.579237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.579219    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 16:17:16.581332 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.581319    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 16:17:16.581374 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.581349    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 16:17:16.583863 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583851    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583869    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583876    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583882    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583892    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583897    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583910    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 16:17:16.583912 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583915    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 16:17:16.584087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583921    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 16:17:16.584087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583927    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 16:17:16.584087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583940    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 16:17:16.584087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.583945    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 16:17:16.584466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.584455    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 16:17:16.584466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.584466    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 16:17:16.588191 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.588179    2566 server.go:1268] "Started kubelet"
Sep 11 16:17:16.588260 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.588238    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 16:17:16.588373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.588312    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 16:17:16.588467 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.588419    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 16:17:16.589044 ip-10-0-128-17 systemd[1]: Started Kubernetes Kubelet.
Sep 11 16:17:16.589316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.589086    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 16:17:16.589585 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.589559    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 16:17:16.590303 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.590282    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 16:17:16.593946 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.593915    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 16:17:16.594720 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.594700    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 16:17:16.596618 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.596592    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 16:17:16.598013 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.596921    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-17.ec2.internal.18d45079ee0efff9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-17.ec2.internal,UID:ip-10-0-128-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-17.ec2.internal,},FirstTimestamp:2026-09-11 16:17:16.588154873 +0000 UTC m=+0.381196660,LastTimestamp:2026-09-11 16:17:16.588154873 +0000 UTC m=+0.381196660,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-17.ec2.internal,}"
Sep 11 16:17:16.599098 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.599083    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 16:17:16.599610 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.599593    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 16:17:16.599696 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.599670    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:16.600171 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600153    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 16:17:16.600266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600183    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 16:17:16.600266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600215    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 16:17:16.600344 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600291    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 16:17:16.600344 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600302    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 16:17:16.600419 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600371    2566 factory.go:55] Registering systemd factory
Sep 11 16:17:16.600419 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600382    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 11 16:17:16.600605 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600593    2566 factory.go:153] Registering CRI-O factory
Sep 11 16:17:16.600659 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600607    2566 factory.go:223] Registration of the crio container factory successfully
Sep 11 16:17:16.600688 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600658    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 16:17:16.600688 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600677    2566 factory.go:103] Registering Raw factory
Sep 11 16:17:16.600760 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.600691    2566 manager.go:1196] Started watching for new ooms in manager
Sep 11 16:17:16.601036 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.601026    2566 manager.go:349] Starting recovery of all containers
Sep 11 16:17:16.605864 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.605834    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 16:17:16.605864 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.605852    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 16:17:16.610958 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.610822    2566 manager.go:354] Recovery completed
Sep 11 16:17:16.615131 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.615116    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.616690 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.616663    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.616763 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.616700    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.616763 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.616711    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.617187 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.617164    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 16:17:16.617187 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.617184    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 16:17:16.617187 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.617185    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.617363 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.617198    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:17:16.618256 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.618238    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.618331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.618270    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.618331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.618286    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.618507 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.618429    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-17.ec2.internal.18d45079efc26173  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-17.ec2.internal,UID:ip-10-0-128-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-17.ec2.internal,},FirstTimestamp:2026-09-11 16:17:16.616687987 +0000 UTC m=+0.409729772,LastTimestamp:2026-09-11 16:17:16.616687987 +0000 UTC m=+0.409729772,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-17.ec2.internal,}"
Sep 11 16:17:16.619218 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.619197    2566 policy_none.go:49] "None policy: Start"
Sep 11 16:17:16.619329 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.619223    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 16:17:16.619329 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.619239    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 16:17:16.621203 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.621190    2566 policy_none.go:47] "Start"
Sep 11 16:17:16.625464 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.625443    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9kn2"
Sep 11 16:17:16.628008 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.627947    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-17.ec2.internal.18d45079efc2a1d4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-17.ec2.internal,UID:ip-10-0-128-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-17.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-17.ec2.internal,},FirstTimestamp:2026-09-11 16:17:16.616704468 +0000 UTC m=+0.409746256,LastTimestamp:2026-09-11 16:17:16.616704468 +0000 UTC m=+0.409746256,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-17.ec2.internal,}"
Sep 11 16:17:16.631519 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.631498    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9kn2"
Sep 11 16:17:16.637122 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.637049    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-17.ec2.internal.18d45079efc2d323  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-17.ec2.internal,UID:ip-10-0-128-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-17.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-17.ec2.internal,},FirstTimestamp:2026-09-11 16:17:16.616717091 +0000 UTC m=+0.409758880,LastTimestamp:2026-09-11 16:17:16.616717091 +0000 UTC m=+0.409758880,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-17.ec2.internal,}"
Sep 11 16:17:16.651044 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651029    2566 manager.go:337] "Starting Device Plugin manager"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.651066    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651080    2566 server.go:87] "Starting device plugin registration server"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651334    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651345    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651458    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651544    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.651554    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.652121    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 16:17:16.674228 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.652175    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:16.726472 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.726448    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 16:17:16.727628 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.727607    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 16:17:16.727716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.727659    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 16:17:16.727716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.727683    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 16:17:16.727817 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.727770    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 16:17:16.730066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.730043    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:16.751815 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.751791    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.752586 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.752570    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.752670 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.752604    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.752670 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.752621    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.752670 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.752651    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.762086 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.762072    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.762147 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.762097    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-17.ec2.internal\": node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:16.789655 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.789594    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:16.828314 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.828284    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"]
Sep 11 16:17:16.828382 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.828358    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.829691 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.829676    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.829776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.829707    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.829776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.829721    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.832077 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832062    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.832187 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832173    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.832246 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832215    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.832847 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832832    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.832847 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832843    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.832984 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832857    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.832984 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832866    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.832984 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832869    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.832984 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.832884    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.835034 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.835021    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.835091 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.835046    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:16.835690 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.835669    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:16.835765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.835701    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:16.835765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.835717    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:16.866999 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.866979    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-17.ec2.internal\" not found" node="ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.871209 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.871194    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-17.ec2.internal\" not found" node="ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.890198 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.890179    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:16.902147 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.902132    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.902242 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.902203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.902242 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:16.902223    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64a62c92ee6be90122c38a626910285c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-17.ec2.internal\" (UID: \"64a62c92ee6be90122c38a626910285c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:16.990905 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:16.990886    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:17.003296 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.003354 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003302    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.003354 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003326    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64a62c92ee6be90122c38a626910285c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-17.ec2.internal\" (UID: \"64a62c92ee6be90122c38a626910285c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.003441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003361    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/64a62c92ee6be90122c38a626910285c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-17.ec2.internal\" (UID: \"64a62c92ee6be90122c38a626910285c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.003441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003372    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.003441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.003384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a08767298a71663b93648f80531fe9e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal\" (UID: \"a08767298a71663b93648f80531fe9e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.091731 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:17.091676    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:17.169109 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.169083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:17.174005 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:17.173987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:18.570210 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:18.570187    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:18.670719 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:18.670684    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:18.771196 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:18.771134    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:18.871633 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:18.871598    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:18.914039 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:18.914015    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 16:17:18.914435 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:18.914137    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:17:18.944636 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:18.944614    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:18.972092 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:18.972067    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:18.978687 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:18.978666    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:19.013223 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.013195    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 16:12:18 +0000 UTC" deadline="2028-05-14 17:52:35.764071119 +0000 UTC"
Sep 11 16:17:19.013223 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.013222    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14665h35m16.750852331s"
Sep 11 16:17:19.072512 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.072490    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:19.103025 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.103005    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:19.104015 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.103998    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:19.104103 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.104034    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:19.104103 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.104051    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:19.145736 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:19.145703    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a62c92ee6be90122c38a626910285c.slice/crio-8b546ad93ef26b20a779ee3347e6d7ab00f3b15e11596fdd30c4ecd0763e2d8a WatchSource:0}: Error finding container 8b546ad93ef26b20a779ee3347e6d7ab00f3b15e11596fdd30c4ecd0763e2d8a: Status 404 returned error can't find the container with id 8b546ad93ef26b20a779ee3347e6d7ab00f3b15e11596fdd30c4ecd0763e2d8a
Sep 11 16:17:19.147467 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:19.147445    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08767298a71663b93648f80531fe9e9.slice/crio-cf3d5db41b9b7d442596f2b040a130d0e5002e5dc1b60221ba737507053c7ffb WatchSource:0}: Error finding container cf3d5db41b9b7d442596f2b040a130d0e5002e5dc1b60221ba737507053c7ffb: Status 404 returned error can't find the container with id cf3d5db41b9b7d442596f2b040a130d0e5002e5dc1b60221ba737507053c7ffb
Sep 11 16:17:19.151707 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.151694    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:17:19.173288 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.173264    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:19.273732 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.273708    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:19.374231 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.374166    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-17.ec2.internal\" not found"
Sep 11 16:17:19.378831 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.378265    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:19.477356 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.477336    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:19.487974 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.487947    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:17:19.488939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.488924    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal"
Sep 11 16:17:19.504567 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.504546    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:17:19.955056 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.955022    2566 apiserver.go:52] "Watching apiserver"
Sep 11 16:17:19.965752 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.965726    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 16:17:19.966275 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.966242    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-qmtsv","kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jk6j8","openshift-image-registry/node-ca-97z2j","openshift-multus/multus-additional-cni-plugins-wznqf","openshift-network-operator/iptables-alerter-pw92x","openshift-ovn-kubernetes/ovnkube-node-nh8vf","kube-system/global-pull-secret-syncer-vcf56","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp","openshift-dns/node-resolver-rbxbl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal","openshift-multus/multus-n78rn","openshift-multus/network-metrics-daemon-dmr9d","openshift-network-diagnostics/network-check-target-87g4d"]
Sep 11 16:17:19.969578 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.969150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.972173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.971447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:19.972173 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.971519    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:19.972173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.972138    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.972363 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.972321    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 16:17:19.973300 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.973257    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xdk5b\""
Sep 11 16:17:19.974676 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.974656    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 16:17:19.974873 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.974857    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 16:17:19.975099 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.975083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.975271 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.975255    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 16:17:19.976014 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.975991    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.976723 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.976704    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:19.977295 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.977277    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 16:17:19.978639 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.978621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.979346 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979157    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dd5v6\""
Sep 11 16:17:19.979346 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979195    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.979507 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979491    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.979817 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979645    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.979817 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979685    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 16:17:19.979974 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.979817    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.980247 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.980233    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fh5bs\""
Sep 11 16:17:19.982089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.981251    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:19.982282 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.981571    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 16:17:19.982954 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.982935    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 16:17:19.983305 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.983288    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.983496 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.983481    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xspvg\""
Sep 11 16:17:19.984152 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.984131    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.984405 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.984386    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 16:17:19.984518 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.984501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:19.985355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.985338    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 16:17:19.985623 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.985607    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:19.986951 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.986929    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plqzg\""
Sep 11 16:17:19.990400 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-log-socket\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.990506 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990425    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.990506 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-systemd\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990621 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-lib-modules\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990621 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.990716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-kubernetes\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-modprobe-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990701    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysconfig\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990859 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-var-lib-kubelet\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.990859 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-binary-copy\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.991000 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.990849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-kubelet\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991062 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-ovn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991113 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-node-log\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52603598-ccf0-4a97-b81b-101edb0d16b4-ovn-node-metrics-cert\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991163 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-conf\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991258 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-systemd-units\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991258 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-netd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991354 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991284    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-config\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991354 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991316    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqgn\" (UniqueName: \"kubernetes.io/projected/52603598-ccf0-4a97-b81b-101edb0d16b4-kube-api-access-dwqgn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991354 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:19.991492 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-host\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991541 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-tmp\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991541 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991522    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991635 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-run\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991635 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991616    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.991733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgkhg\" (UniqueName: \"kubernetes.io/projected/4c64e971-ff67-44a8-959e-48fc454e6d36-kube-api-access-vgkhg\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.991733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991672    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-slash\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-netns\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991870 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991870 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-bin\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991870 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-env-overrides\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.991870 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-sys\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.991870 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-etc-tuned\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.992124 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-systemd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.992124 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.991982    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:19.992124 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-var-lib-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992127    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-etc-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992148    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992186    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqzx\" (UniqueName: \"kubernetes.io/projected/57db61b1-242f-476a-b2be-d0636b8328b5-kube-api-access-4wqzx\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992214    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ce9172b-8309-4a61-980b-f98dbcdf1da4-host\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:19.992265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhkx\" (UniqueName: \"kubernetes.io/projected/9ce9172b-8309-4a61-980b-f98dbcdf1da4-kube-api-access-hxhkx\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-system-cni-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-script-lib\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbrdb\" (UniqueName: \"kubernetes.io/projected/61003777-8036-4800-b51c-8eb33a3efb4f-kube-api-access-pbrdb\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992355    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ce9172b-8309-4a61-980b-f98dbcdf1da4-serviceca\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992360    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-cnibin\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.992063    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992496    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.992537 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.992942 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.992568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:19.993069 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.993053    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.993249 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.993232    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h5fcc\""
Sep 11 16:17:19.993309 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.993248    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pvbm2\""
Sep 11 16:17:19.993913 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.993854    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 16:17:19.994089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.994039    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 16:17:19.995525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.995209    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.997685    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.997992    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.998211    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mj2s9\""
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.998433    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:19.998491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:19.998672 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:19.998555    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:20.001222 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.001202    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.004643 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.004422    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 16:17:20.004643 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.004467    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wxpqb\""
Sep 11 16:17:20.018121 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.018096    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z7s79"
Sep 11 16:17:20.027773 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.027754    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z7s79"
Sep 11 16:17:20.079095 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.079024    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.092983    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-bin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-multus\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093060    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-multus-certs\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-log-socket\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093113    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-lib-modules\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093163    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e137e4d3-027f-4a47-9de3-018983101c0b-iptables-alerter-script\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093186    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-socket-dir-parent\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-kubernetes\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-modprobe-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysconfig\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-var-lib-kubelet\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-konnectivity-ca\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093351    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-cni-binary-copy\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-ovn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.096215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-lib-modules\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093455    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52603598-ccf0-4a97-b81b-101edb0d16b4-ovn-node-metrics-cert\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093535    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-log-socket\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-conf\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093585    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysconfig\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093590    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-systemd-units\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-netd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqgn\" (UniqueName: \"kubernetes.io/projected/52603598-ccf0-4a97-b81b-101edb0d16b4-kube-api-access-dwqgn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093776    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-modprobe-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-host\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-tmp\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093829    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-var-lib-kubelet\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-kubelet-config\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-hostroot\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-slash\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097110 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093943    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-bin\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093980    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-etc-tuned\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ce9172b-8309-4a61-980b-f98dbcdf1da4-host\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094028    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-var-lib-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqzx\" (UniqueName: \"kubernetes.io/projected/57db61b1-242f-476a-b2be-d0636b8328b5-kube-api-access-4wqzx\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094107    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhkx\" (UniqueName: \"kubernetes.io/projected/9ce9172b-8309-4a61-980b-f98dbcdf1da4-kube-api-access-hxhkx\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-cnibin\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-script-lib\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-system-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094303    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 16:17:20.097907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-systemd\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094419    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhnrn\" (UniqueName: \"kubernetes.io/projected/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kube-api-access-zhnrn\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-k8s-cni-cncf-io\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-netns\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-etc-kubernetes\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094533    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-binary-copy\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e137e4d3-027f-4a47-9de3-018983101c0b-host-slash\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh5z8\" (UniqueName: \"kubernetes.io/projected/e137e4d3-027f-4a47-9de3-018983101c0b-kube-api-access-nh5z8\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094608    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-conf-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094646    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-kubelet\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094689    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-node-log\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094708    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-systemd-units\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094751    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-netd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.101417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-daemon-config\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-config\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094831    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-dbus\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-cnibin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.094901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-run\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095028    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095035    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-cnibin\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.093709    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-kubernetes\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095275    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-slash\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.095351    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.095438    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:20.595403023 +0000 UTC m=+3.010885987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgkhg\" (UniqueName: \"kubernetes.io/projected/4c64e971-ff67-44a8-959e-48fc454e6d36-kube-api-access-vgkhg\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095555    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-kubelet\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095658    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-host\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.102373 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-cni-bin\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095809    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-script-lib\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-device-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-netns\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-env-overrides\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096423    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-sys\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096451    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-system-cni-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096478    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqbqj\" (UniqueName: \"kubernetes.io/projected/33c04e0f-899f-42a0-8f19-a6725cae80df-kube-api-access-tqbqj\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096531    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-systemd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-etc-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-agent-certs\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096606    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b554964-07b2-4935-9234-e36967e235d7-hosts-file\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbrdb\" (UniqueName: \"kubernetes.io/projected/61003777-8036-4800-b51c-8eb33a3efb4f-kube-api-access-pbrdb\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ce9172b-8309-4a61-980b-f98dbcdf1da4-serviceca\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b554964-07b2-4935-9234-e36967e235d7-tmp-dir\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wb4p\" (UniqueName: \"kubernetes.io/projected/4b554964-07b2-4935-9234-e36967e235d7-kube-api-access-2wb4p\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096786    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-netns\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.103252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.096973    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-ovnkube-config\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.095236    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-ovn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52603598-ccf0-4a97-b81b-101edb0d16b4-env-overrides\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097233    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ce9172b-8309-4a61-980b-f98dbcdf1da4-host\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-var-lib-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097308    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097350    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-systemd\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-etc-openvswitch\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-sys\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097587    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c64e971-ff67-44a8-959e-48fc454e6d36-system-cni-dir\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097623    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-run-systemd\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-conf\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-host-kubelet\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097773    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097793    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52603598-ccf0-4a97-b81b-101edb0d16b4-node-log\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-etc-sysctl-d\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097942    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-binary-copy\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.097990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61003777-8036-4800-b51c-8eb33a3efb4f-run\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.107283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.098161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ce9172b-8309-4a61-980b-f98dbcdf1da4-serviceca\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.099389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-tmp\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.099425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61003777-8036-4800-b51c-8eb33a3efb4f-etc-tuned\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.101091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52603598-ccf0-4a97-b81b-101edb0d16b4-ovn-node-metrics-cert\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.101385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c64e971-ff67-44a8-959e-48fc454e6d36-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.106701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgkhg\" (UniqueName: \"kubernetes.io/projected/4c64e971-ff67-44a8-959e-48fc454e6d36-kube-api-access-vgkhg\") pod \"multus-additional-cni-plugins-wznqf\" (UID: \"4c64e971-ff67-44a8-959e-48fc454e6d36\") " pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.108189 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.107062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqgn\" (UniqueName: \"kubernetes.io/projected/52603598-ccf0-4a97-b81b-101edb0d16b4-kube-api-access-dwqgn\") pod \"ovnkube-node-nh8vf\" (UID: \"52603598-ccf0-4a97-b81b-101edb0d16b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.108468 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.108265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqzx\" (UniqueName: \"kubernetes.io/projected/57db61b1-242f-476a-b2be-d0636b8328b5-kube-api-access-4wqzx\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:20.108723 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.108687    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhkx\" (UniqueName: \"kubernetes.io/projected/9ce9172b-8309-4a61-980b-f98dbcdf1da4-kube-api-access-hxhkx\") pod \"node-ca-97z2j\" (UID: \"9ce9172b-8309-4a61-980b-f98dbcdf1da4\") " pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.109626 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.109606    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbrdb\" (UniqueName: \"kubernetes.io/projected/61003777-8036-4800-b51c-8eb33a3efb4f-kube-api-access-pbrdb\") pod \"tuned-jk6j8\" (UID: \"61003777-8036-4800-b51c-8eb33a3efb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.112805 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.112754    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal" event={"ID":"a08767298a71663b93648f80531fe9e9","Type":"ContainerStarted","Data":"cf3d5db41b9b7d442596f2b040a130d0e5002e5dc1b60221ba737507053c7ffb"}
Sep 11 16:17:20.114601 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.114577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal" event={"ID":"64a62c92ee6be90122c38a626910285c","Type":"ContainerStarted","Data":"8b546ad93ef26b20a779ee3347e6d7ab00f3b15e11596fdd30c4ecd0763e2d8a"}
Sep 11 16:17:20.197241 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197209    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-agent-certs\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.197378 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b554964-07b2-4935-9234-e36967e235d7-hosts-file\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.197378 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197328    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b554964-07b2-4935-9234-e36967e235d7-hosts-file\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.197465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b554964-07b2-4935-9234-e36967e235d7-tmp-dir\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.197465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wb4p\" (UniqueName: \"kubernetes.io/projected/4b554964-07b2-4935-9234-e36967e235d7-kube-api-access-2wb4p\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.197465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-bin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-multus\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-multus-certs\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e137e4d3-027f-4a47-9de3-018983101c0b-iptables-alerter-script\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-socket-dir-parent\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-multus\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197611    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-konnectivity-ca\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-cni-binary-copy\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-cni-bin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197673    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197695    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b554964-07b2-4935-9234-e36967e235d7-tmp-dir\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-kubelet-config\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.197736    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-kubelet-config\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197801    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-hostroot\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.197815    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:20.69779681 +0000 UTC m=+3.113279773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-hostroot\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-multus-certs\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.197926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-system-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.197942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhnrn\" (UniqueName: \"kubernetes.io/projected/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kube-api-access-zhnrn\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-k8s-cni-cncf-io\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198078    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198087    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-k8s-cni-cncf-io\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-konnectivity-ca\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198141    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-socket-dir-parent\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-system-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198215    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-netns\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-etc-kubernetes\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e137e4d3-027f-4a47-9de3-018983101c0b-host-slash\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh5z8\" (UniqueName: \"kubernetes.io/projected/e137e4d3-027f-4a47-9de3-018983101c0b-kube-api-access-nh5z8\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-conf-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198333    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-cni-binary-copy\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.198750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-etc-kubernetes\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-daemon-config\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198406    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-conf-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-dbus\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-cnibin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-run-netns\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-kubelet\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/913732e3-f194-491d-9b20-84dcaf859bff-dbus\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-cnibin\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198420    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-cni-dir\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e137e4d3-027f-4a47-9de3-018983101c0b-iptables-alerter-script\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198561    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-device-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e137e4d3-027f-4a47-9de3-018983101c0b-host-slash\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c04e0f-899f-42a0-8f19-a6725cae80df-host-var-lib-kubelet\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqbqj\" (UniqueName: \"kubernetes.io/projected/33c04e0f-899f-42a0-8f19-a6725cae80df-kube-api-access-tqbqj\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.199426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-device-dir\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.200256 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.198848    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33c04e0f-899f-42a0-8f19-a6725cae80df-multus-daemon-config\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.200256 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.199935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7598365-5a86-4eb8-8fb0-18d230ebf3aa-agent-certs\") pod \"konnectivity-agent-qmtsv\" (UID: \"a7598365-5a86-4eb8-8fb0-18d230ebf3aa\") " pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.203977 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.203943    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:20.204367 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.204347    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:20.204462 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.204373    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:20.204462 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.204387    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:20.204462 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.204434    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:20.704421929 +0000 UTC m=+3.119904890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:20.209590 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.209543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wb4p\" (UniqueName: \"kubernetes.io/projected/4b554964-07b2-4935-9234-e36967e235d7-kube-api-access-2wb4p\") pod \"node-resolver-rbxbl\" (UID: \"4b554964-07b2-4935-9234-e36967e235d7\") " pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.215377 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.215313    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhnrn\" (UniqueName: \"kubernetes.io/projected/5b478f99-7867-433b-a8d8-a1fe39ca3fc4-kube-api-access-zhnrn\") pod \"aws-ebs-csi-driver-node-7w7sp\" (UID: \"5b478f99-7867-433b-a8d8-a1fe39ca3fc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.215470 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.215399    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqbqj\" (UniqueName: \"kubernetes.io/projected/33c04e0f-899f-42a0-8f19-a6725cae80df-kube-api-access-tqbqj\") pod \"multus-n78rn\" (UID: \"33c04e0f-899f-42a0-8f19-a6725cae80df\") " pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.215558 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.215543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh5z8\" (UniqueName: \"kubernetes.io/projected/e137e4d3-027f-4a47-9de3-018983101c0b-kube-api-access-nh5z8\") pod \"iptables-alerter-pw92x\" (UID: \"e137e4d3-027f-4a47-9de3-018983101c0b\") " pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.287117 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.287086    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:20.299489 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.299468    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jk6j8"
Sep 11 16:17:20.310101 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.310079    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97z2j"
Sep 11 16:17:20.314953 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.314934    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wznqf"
Sep 11 16:17:20.322492 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.322475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pw92x"
Sep 11 16:17:20.332079 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.332062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp"
Sep 11 16:17:20.338722 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.338699    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:20.346255 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.346238    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rbxbl"
Sep 11 16:17:20.351821 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.351805    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n78rn"
Sep 11 16:17:20.600563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.600524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:20.600718 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.600677    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:20.600779 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.600740    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:21.600725326 +0000 UTC m=+4.016208290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:20.701801 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.701770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:20.701980 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.701895    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:20.701980 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.701976    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:21.701943936 +0000 UTC m=+4.117426896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:20.802757 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:20.802722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:20.802914 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.802897    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:20.802983 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.802919    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:20.802983 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.802932    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:20.803096 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:20.803010    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:21.802993671 +0000 UTC m=+4.218476635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:20.987067 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:20.986904    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b478f99_7867_433b_a8d8_a1fe39ca3fc4.slice/crio-3fd6786439f7857d187d2a8aa163145ab9bcce36febdf807b2a394fb5ee51b7c WatchSource:0}: Error finding container 3fd6786439f7857d187d2a8aa163145ab9bcce36febdf807b2a394fb5ee51b7c: Status 404 returned error can't find the container with id 3fd6786439f7857d187d2a8aa163145ab9bcce36febdf807b2a394fb5ee51b7c
Sep 11 16:17:20.990115 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:20.990088    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c04e0f_899f_42a0_8f19_a6725cae80df.slice/crio-7f4adff161bb2d7cd4ccbefd9adef028e9b273531cdbe188087158411f833c57 WatchSource:0}: Error finding container 7f4adff161bb2d7cd4ccbefd9adef028e9b273531cdbe188087158411f833c57: Status 404 returned error can't find the container with id 7f4adff161bb2d7cd4ccbefd9adef028e9b273531cdbe188087158411f833c57
Sep 11 16:17:20.992188 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:20.992151    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce9172b_8309_4a61_980b_f98dbcdf1da4.slice/crio-0f91fe17f40dcc395211b02482ea8e128c7894b010ab0405c4d8471bd42f2ae3 WatchSource:0}: Error finding container 0f91fe17f40dcc395211b02482ea8e128c7894b010ab0405c4d8471bd42f2ae3: Status 404 returned error can't find the container with id 0f91fe17f40dcc395211b02482ea8e128c7894b010ab0405c4d8471bd42f2ae3
Sep 11 16:17:20.992946 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:20.992927    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61003777_8036_4800_b51c_8eb33a3efb4f.slice/crio-c26f4ecc55312f6ab804119549b9debb02c0ebe555443ee7767957638901dded WatchSource:0}: Error finding container c26f4ecc55312f6ab804119549b9debb02c0ebe555443ee7767957638901dded: Status 404 returned error can't find the container with id c26f4ecc55312f6ab804119549b9debb02c0ebe555443ee7767957638901dded
Sep 11 16:17:21.014953 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:21.014928    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode137e4d3_027f_4a47_9de3_018983101c0b.slice/crio-e7e061b99b0473fdb124b393fe93c03c68d375e61ea2f3712c8462f56d262033 WatchSource:0}: Error finding container e7e061b99b0473fdb124b393fe93c03c68d375e61ea2f3712c8462f56d262033: Status 404 returned error can't find the container with id e7e061b99b0473fdb124b393fe93c03c68d375e61ea2f3712c8462f56d262033
Sep 11 16:17:21.015816 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:21.015796    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c64e971_ff67_44a8_959e_48fc454e6d36.slice/crio-8c9cbc862cc5aa7fb238b0fd2b2cb0339a15f614e6a23c39cc581e9c0e041760 WatchSource:0}: Error finding container 8c9cbc862cc5aa7fb238b0fd2b2cb0339a15f614e6a23c39cc581e9c0e041760: Status 404 returned error can't find the container with id 8c9cbc862cc5aa7fb238b0fd2b2cb0339a15f614e6a23c39cc581e9c0e041760
Sep 11 16:17:21.016470 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:21.016416    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52603598_ccf0_4a97_b81b_101edb0d16b4.slice/crio-eccd6417e260cc0ae53554de15de4f1c81771b204d8593431c007c9c2738215f WatchSource:0}: Error finding container eccd6417e260cc0ae53554de15de4f1c81771b204d8593431c007c9c2738215f: Status 404 returned error can't find the container with id eccd6417e260cc0ae53554de15de4f1c81771b204d8593431c007c9c2738215f
Sep 11 16:17:21.017393 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:21.017292    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b554964_07b2_4935_9234_e36967e235d7.slice/crio-7ae2943e6df111e6f7b3061c21361dde769833e8f5d8494c906fb34f0fc772d6 WatchSource:0}: Error finding container 7ae2943e6df111e6f7b3061c21361dde769833e8f5d8494c906fb34f0fc772d6: Status 404 returned error can't find the container with id 7ae2943e6df111e6f7b3061c21361dde769833e8f5d8494c906fb34f0fc772d6
Sep 11 16:17:21.017978 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:21.017937    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7598365_5a86_4eb8_8fb0_18d230ebf3aa.slice/crio-54d2bb3eefcaf9acc01c4d45c1634afa059d9faffce840ea8eb66bfa383c8cac WatchSource:0}: Error finding container 54d2bb3eefcaf9acc01c4d45c1634afa059d9faffce840ea8eb66bfa383c8cac: Status 404 returned error can't find the container with id 54d2bb3eefcaf9acc01c4d45c1634afa059d9faffce840ea8eb66bfa383c8cac
Sep 11 16:17:21.031416 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.031385    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:20 +0000 UTC" deadline="2028-04-03 13:18:05.98260198 +0000 UTC"
Sep 11 16:17:21.031502 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.031417    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13677h0m44.951189745s"
Sep 11 16:17:21.105462 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.105443    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:21.105580 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.105557    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:21.119099 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.119076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rbxbl" event={"ID":"4b554964-07b2-4935-9234-e36967e235d7","Type":"ContainerStarted","Data":"7ae2943e6df111e6f7b3061c21361dde769833e8f5d8494c906fb34f0fc772d6"}
Sep 11 16:17:21.121225 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.121204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"eccd6417e260cc0ae53554de15de4f1c81771b204d8593431c007c9c2738215f"}
Sep 11 16:17:21.122458 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.122436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerStarted","Data":"8c9cbc862cc5aa7fb238b0fd2b2cb0339a15f614e6a23c39cc581e9c0e041760"}
Sep 11 16:17:21.125606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.125580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pw92x" event={"ID":"e137e4d3-027f-4a47-9de3-018983101c0b","Type":"ContainerStarted","Data":"e7e061b99b0473fdb124b393fe93c03c68d375e61ea2f3712c8462f56d262033"}
Sep 11 16:17:21.129143 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.129120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jk6j8" event={"ID":"61003777-8036-4800-b51c-8eb33a3efb4f","Type":"ContainerStarted","Data":"c26f4ecc55312f6ab804119549b9debb02c0ebe555443ee7767957638901dded"}
Sep 11 16:17:21.130449 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.130430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp" event={"ID":"5b478f99-7867-433b-a8d8-a1fe39ca3fc4","Type":"ContainerStarted","Data":"3fd6786439f7857d187d2a8aa163145ab9bcce36febdf807b2a394fb5ee51b7c"}
Sep 11 16:17:21.134653 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.134614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal" event={"ID":"64a62c92ee6be90122c38a626910285c","Type":"ContainerStarted","Data":"78302b3d2d46ab468a3a0d2bec99ad80ffe7f3fb1a702d4775ea2f2c66c9050e"}
Sep 11 16:17:21.135785 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.135766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmtsv" event={"ID":"a7598365-5a86-4eb8-8fb0-18d230ebf3aa","Type":"ContainerStarted","Data":"54d2bb3eefcaf9acc01c4d45c1634afa059d9faffce840ea8eb66bfa383c8cac"}
Sep 11 16:17:21.138593 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.138573    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97z2j" event={"ID":"9ce9172b-8309-4a61-980b-f98dbcdf1da4","Type":"ContainerStarted","Data":"0f91fe17f40dcc395211b02482ea8e128c7894b010ab0405c4d8471bd42f2ae3"}
Sep 11 16:17:21.141449 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.141420    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n78rn" event={"ID":"33c04e0f-899f-42a0-8f19-a6725cae80df","Type":"ContainerStarted","Data":"7f4adff161bb2d7cd4ccbefd9adef028e9b273531cdbe188087158411f833c57"}
Sep 11 16:17:21.150567 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.150526    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-17.ec2.internal" podStartSLOduration=2.150515073 podStartE2EDuration="2.150515073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:21.150423328 +0000 UTC m=+3.565906309" watchObservedRunningTime="2026-09-11 16:17:21.150515073 +0000 UTC m=+3.565998050"
Sep 11 16:17:21.607711 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.607070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:21.607711 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.607202    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:21.607711 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.607268    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:23.607249549 +0000 UTC m=+6.022732515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:21.708573 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.708166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:21.708573 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.708352    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:21.708573 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.708415    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:23.708394586 +0000 UTC m=+6.123877556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:21.808973 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:21.808924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:21.809137 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.809085    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:21.809137 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.809108    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:21.809137 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.809121    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:21.809329 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:21.809177    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:23.809160749 +0000 UTC m=+6.224643723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:22.032773 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.032563    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:20 +0000 UTC" deadline="2028-05-19 09:00:29.994907162 +0000 UTC"
Sep 11 16:17:22.032773 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.032615    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14776h43m7.96229553s"
Sep 11 16:17:22.107689 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.107654    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:22.107866 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:22.107784    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:22.108280 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.108257    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:22.108382 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:22.108362    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:22.163758 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.163727    2566 generic.go:356] "Generic (PLEG): container finished" podID="a08767298a71663b93648f80531fe9e9" containerID="e87454ee4c93af7a7891c7b7704b0828df650200de75b76c05c97ed1f3157f20" exitCode=0
Sep 11 16:17:22.163894 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:22.163811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal" event={"ID":"a08767298a71663b93648f80531fe9e9","Type":"ContainerDied","Data":"e87454ee4c93af7a7891c7b7704b0828df650200de75b76c05c97ed1f3157f20"}
Sep 11 16:17:23.106217 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:23.106124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:23.106865 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.106411    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:23.185771 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:23.185690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal" event={"ID":"a08767298a71663b93648f80531fe9e9","Type":"ContainerStarted","Data":"197359927cc23dabbdac904ccb63d16240318b0a869b63831b7efc959c7ee06f"}
Sep 11 16:17:23.625514 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:23.625478    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:23.625685 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.625619    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:23.625685 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.625678    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.625660235 +0000 UTC m=+10.041143209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:23.726066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:23.726032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:23.726241 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.726168    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:23.726241 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.726235    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.726217396 +0000 UTC m=+10.141700370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:23.827089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:23.827044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:23.827286 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.827262    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:23.827286 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.827282    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:23.827411 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.827295    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:23.827411 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:23.827355    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.827335859 +0000 UTC m=+10.242818825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:24.105713 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:24.105683    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:24.105713 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:24.105708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:24.105946 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:24.105825    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:24.106019 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:24.105935    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:25.106418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:25.106372    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:25.106866 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:25.106518    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:26.106204 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:26.105501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:26.106204 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:26.105623    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:26.106204 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:26.105452    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:26.106204 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:26.106120    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:27.106312 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:27.106280    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:27.106540 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.106512    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:27.659804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:27.659764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:27.659989 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.659974    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:27.660119 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.660051    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:35.660029141 +0000 UTC m=+18.075512105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:27.760976 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:27.760924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:27.761135 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.761082    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:27.761209 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.761150    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:35.761133499 +0000 UTC m=+18.176616461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:27.861559 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:27.861523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:27.861736 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.861716    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:27.861804 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.861745    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:27.861804 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.861760    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:27.861902 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:27.861826    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:35.861805705 +0000 UTC m=+18.277288678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:28.107328 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:28.107115    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:28.107328 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:28.107236    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:28.107855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:28.107611    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:28.107855 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:28.107706    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:29.105860 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:29.105770    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:29.106055 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:29.105913    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:30.106053 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:30.106019    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:30.106053 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:30.106050    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:30.106547 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:30.106147    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:30.106547 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:30.106285    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:31.106061 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:31.106031    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:31.106408 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:31.106128    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:32.106134 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:32.106098    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:32.106550 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:32.106102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:32.106550 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:32.106230    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:32.106550 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:32.106331    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:33.105695 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:33.105664    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:33.105869 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:33.105782    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:34.106412 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:34.106171    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:34.106850 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:34.106219    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:34.106850 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:34.106506    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:34.106850 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:34.106594    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:35.105944 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:35.105917    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:35.106168 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.106031    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:35.719916 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:35.719876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:35.720341 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.720040    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:35.720341 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.720107    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:51.720092243 +0000 UTC m=+34.135575201 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:35.821171 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:35.821137    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:35.821351 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.821303    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:35.821403 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.821372    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:17:51.821353471 +0000 UTC m=+34.236836440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:35.921947 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:35.921912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:35.922117 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.922064    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:35.922117 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.922084    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:35.922117 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.922093    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:35.922243 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:35.922140    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:51.922126648 +0000 UTC m=+34.337609607 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:36.106562 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:36.106523    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:36.106562 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:36.106566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:36.106811 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:36.106647    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:36.106811 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:36.106760    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:37.106092 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:37.106059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:37.106499 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:37.106194    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:38.107099 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:38.107059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:38.107558 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:38.107108    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:38.107558 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:38.107185    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:38.107558 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:38.107299    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:39.106704 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.106524    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:39.106823 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:39.106800    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:39.228112 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.228075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rbxbl" event={"ID":"4b554964-07b2-4935-9234-e36967e235d7","Type":"ContainerStarted","Data":"8b6f63c45e034979b278ffa28ce9890dd34cab985ce9266b9704a5e096f6b7c1"}
Sep 11 16:17:39.231143 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.231116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"ea70ffad24c8bbe367bbbb57b0f90ecb08dd925f74ba9d712cdef4b8294361f9"}
Sep 11 16:17:39.231230 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.231183    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"036f4848b8949f373279727ebef34df67ae7a83fd63de88cc23e02cb1a8d2808"}
Sep 11 16:17:39.231230 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.231215    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"3011b320b37a19b2b02ceea0188bb3f3edf8aaf60f8c6134fa140989e00910b3"}
Sep 11 16:17:39.231314 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.231251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"d4f384e7e34226bd01cf5228c68d6244d8a36d4c8e8f441fcd7ae4e4ec672daf"}
Sep 11 16:17:39.231314 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.231280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"2ac10ab662a034a33b5cd06c6e510e370eb2957104bf55f144dba7fe58153c58"}
Sep 11 16:17:39.232861 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.232839    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="7854d1b3a77575fd52b4baab7befe55bb891eb8a61c6103ad9964d649d5c9ba4" exitCode=0
Sep 11 16:17:39.232974 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.232906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"7854d1b3a77575fd52b4baab7befe55bb891eb8a61c6103ad9964d649d5c9ba4"}
Sep 11 16:17:39.234491 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.234457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jk6j8" event={"ID":"61003777-8036-4800-b51c-8eb33a3efb4f","Type":"ContainerStarted","Data":"7cc46ea70ef686e3f758e6642ba8e175347e6547ca91afe0ffabebd7a91dd812"}
Sep 11 16:17:39.235886 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.235867    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp" event={"ID":"5b478f99-7867-433b-a8d8-a1fe39ca3fc4","Type":"ContainerStarted","Data":"35992a9ace3e4dbc6ffe5551d09a4279b0a53a6e5432a25b9561a4e76b99036f"}
Sep 11 16:17:39.237085 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.237065    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmtsv" event={"ID":"a7598365-5a86-4eb8-8fb0-18d230ebf3aa","Type":"ContainerStarted","Data":"c11c89e579247da80202c532561546067562f4836ba3032b83110926e2447aea"}
Sep 11 16:17:39.238563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.238542    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97z2j" event={"ID":"9ce9172b-8309-4a61-980b-f98dbcdf1da4","Type":"ContainerStarted","Data":"616575a3ef4b111257b3f210206f8cd99bd98fa0460aa7cf1274ff22f4c730b7"}
Sep 11 16:17:39.239944 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.239926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n78rn" event={"ID":"33c04e0f-899f-42a0-8f19-a6725cae80df","Type":"ContainerStarted","Data":"33d5917dd323d247b6b2da0c9b79bce083e0b050b0b1eda5c50b7a45cd0e529f"}
Sep 11 16:17:39.247317 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.247265    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rbxbl" podStartSLOduration=3.552901494 podStartE2EDuration="21.247248278s" totalImagesPullingTime="17.694346784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:39.245791069 +0000 UTC m=+21.661274050" watchObservedRunningTime="2026-09-11 16:17:39.247248278 +0000 UTC m=+21.662731259"
Sep 11 16:17:39.247400 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.247372    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-17.ec2.internal" podStartSLOduration=20.247364309 podStartE2EDuration="20.247364309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:23.203082178 +0000 UTC m=+5.618565160" watchObservedRunningTime="2026-09-11 16:17:39.247364309 +0000 UTC m=+21.662847290"
Sep 11 16:17:39.266570 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.266520    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n78rn" podStartSLOduration=3.36322153 podStartE2EDuration="21.266504933s" totalImagesPullingTime="17.903283403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:39.266397496 +0000 UTC m=+21.681880643" watchObservedRunningTime="2026-09-11 16:17:39.266504933 +0000 UTC m=+21.681987910"
Sep 11 16:17:39.327804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.327756    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qmtsv" podStartSLOduration=3.944825995 podStartE2EDuration="21.327736382s" totalImagesPullingTime="17.382910387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:39.305990488 +0000 UTC m=+21.721473517" watchObservedRunningTime="2026-09-11 16:17:39.327736382 +0000 UTC m=+21.743219364"
Sep 11 16:17:39.327952 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.327857    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jk6j8" podStartSLOduration=3.628810289 podStartE2EDuration="21.327851618s" totalImagesPullingTime="17.699041329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:39.327156373 +0000 UTC m=+21.742639353" watchObservedRunningTime="2026-09-11 16:17:39.327851618 +0000 UTC m=+21.743334615"
Sep 11 16:17:39.351762 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.351711    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-97z2j" podStartSLOduration=3.70061238 podStartE2EDuration="21.351696053s" totalImagesPullingTime="17.651083673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:39.351362202 +0000 UTC m=+21.766845204" watchObservedRunningTime="2026-09-11 16:17:39.351696053 +0000 UTC m=+21.767179037"
Sep 11 16:17:39.519366 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.519326    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:39.519998 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.519951    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:39.840906 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:39.840880    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 16:17:40.042010 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.041884    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T16:17:39.840901094Z","UUID":"003049ee-6620-4e5f-8a8d-c2273784e2be","Handler":null,"Name":"","Endpoint":""}
Sep 11 16:17:40.043619 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.043596    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 16:17:40.043731 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.043636    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 16:17:40.106306 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.106266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:40.106306 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.106304    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:40.106550 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:40.106405    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:40.106550 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:40.106520    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:40.246649 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.246606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"1228d215c530bcd873f3118893a39c0d7383416f1b2f6682a6044b6389fff8c1"}
Sep 11 16:17:40.248525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.248496    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pw92x" event={"ID":"e137e4d3-027f-4a47-9de3-018983101c0b","Type":"ContainerStarted","Data":"7780e417f0df144139f4d5ed3d9fb795ddd05debd24725faa6b1c1c67c755ce6"}
Sep 11 16:17:40.250721 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.250647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp" event={"ID":"5b478f99-7867-433b-a8d8-a1fe39ca3fc4","Type":"ContainerStarted","Data":"813271ad797a03a123812b6565ad34d86171d3b0faa8e058db774d8794d72692"}
Sep 11 16:17:40.253111 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.253032    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:40.254634 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.253289    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qmtsv"
Sep 11 16:17:40.270949 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:40.270894    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pw92x" podStartSLOduration=4.622710324 podStartE2EDuration="22.270878428s" totalImagesPullingTime="17.648168104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:40.270743243 +0000 UTC m=+22.686226237" watchObservedRunningTime="2026-09-11 16:17:40.270878428 +0000 UTC m=+22.686361416"
Sep 11 16:17:41.106240 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:41.106201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:41.106435 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:41.106412    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:41.255797 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:41.255755    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp" event={"ID":"5b478f99-7867-433b-a8d8-a1fe39ca3fc4","Type":"ContainerStarted","Data":"e4ccfe6681b6eed85f0faff35cf7c3c705f18739b28ad27bced6728d9d1c14bc"}
Sep 11 16:17:41.276238 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:41.276174    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7sp" podStartSLOduration=4.031269219 podStartE2EDuration="23.276151555s" totalImagesPullingTime="19.244882336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:41.275197196 +0000 UTC m=+23.690680179" watchObservedRunningTime="2026-09-11 16:17:41.276151555 +0000 UTC m=+23.691634538"
Sep 11 16:17:42.105786 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:42.105745    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:42.105786 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:42.105741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:42.106139 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:42.105874    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:42.106139 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:42.106011    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:42.263395 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:42.263288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"325552f66672c7b5e68e33a4144bca40297d3f890ab00c4e1728767299130518"}
Sep 11 16:17:43.106519 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:43.106473    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:43.106710 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:43.106611    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:44.105999 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.105793    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:44.106509 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.105859    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:44.106509 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:44.106072    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:44.106509 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:44.106161    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:44.271708 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.271671    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" event={"ID":"52603598-ccf0-4a97-b81b-101edb0d16b4","Type":"ContainerStarted","Data":"14c7fc8f09647c08fb6f3b6b68f0efc82f403896f9c02f2ff6703282c8347675"}
Sep 11 16:17:44.271907 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.271879    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:44.271993 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.271951    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:44.273808 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.273783    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="8c5a1963d1ba012d3e1f45c897b84173c0c199e133593ed59900ac3afa763536" exitCode=0
Sep 11 16:17:44.273919 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.273815    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"8c5a1963d1ba012d3e1f45c897b84173c0c199e133593ed59900ac3afa763536"}
Sep 11 16:17:44.288143 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.288116    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:44.302698 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:44.302655    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf" podStartSLOduration=8.571780363 podStartE2EDuration="26.302639515s" totalImagesPullingTime="17.730859152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:44.30230346 +0000 UTC m=+26.717786441" watchObservedRunningTime="2026-09-11 16:17:44.302639515 +0000 UTC m=+26.718122547"
Sep 11 16:17:45.106331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.106303    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:45.107014 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:45.106416    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:45.276499 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.276460    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:45.292610 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.292583    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:17:45.418263 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.418234    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-87g4d"]
Sep 11 16:17:45.418413 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.418370    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:45.418498 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:45.418473    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:45.419946 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.419918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcf56"]
Sep 11 16:17:45.420091 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.420035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:45.420168 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:45.420134    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:45.431892 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.431871    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dmr9d"]
Sep 11 16:17:45.432007 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:45.431995    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:45.432099 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:45.432076    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:46.280600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:46.280563    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="24af443f75e3703d4ada5fb2bed6f030dbfd3a48f2da1f3c81ab9c274ce04f1f" exitCode=0
Sep 11 16:17:46.281208 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:46.280632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"24af443f75e3703d4ada5fb2bed6f030dbfd3a48f2da1f3c81ab9c274ce04f1f"}
Sep 11 16:17:47.105607 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:47.105575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:47.105776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:47.105575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:47.105776 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:47.105672    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:47.105776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:47.105575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:47.105776 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:47.105740    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:47.105912 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:47.105810    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:47.284997 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:47.284944    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="82b433b7bd70bec9b7c940995a9ecaaaeecf8fd4e67e38dc52acc0cb6b2c7b53" exitCode=0
Sep 11 16:17:47.284997 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:47.284987    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"82b433b7bd70bec9b7c940995a9ecaaaeecf8fd4e67e38dc52acc0cb6b2c7b53"}
Sep 11 16:17:49.106588 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:49.106333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:49.107280 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:49.106335    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:49.107280 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:49.106692    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:49.107280 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:49.106335    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:49.107280 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:49.106789    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:49.107280 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:49.106868    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:51.105977 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.105929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:51.105977 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.105952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:51.106569 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.105975    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:51.106569 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.106066    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dmr9d" podUID="57db61b1-242f-476a-b2be-d0636b8328b5"
Sep 11 16:17:51.106569 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.106126    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87g4d" podUID="3136b33e-fd4d-4743-98c0-043730b3d595"
Sep 11 16:17:51.106569 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.106231    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcf56" podUID="913732e3-f194-491d-9b20-84dcaf859bff"
Sep 11 16:17:51.731916 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.731884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:51.732102 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.732015    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:51.732102 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.732067    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs podName:57db61b1-242f-476a-b2be-d0636b8328b5 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.732052429 +0000 UTC m=+66.147535388 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs") pod "network-metrics-daemon-dmr9d" (UID: "57db61b1-242f-476a-b2be-d0636b8328b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:51.832458 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.832429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:51.832571 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.832534    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:51.832610 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.832589    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret podName:913732e3-f194-491d-9b20-84dcaf859bff nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.832575098 +0000 UTC m=+66.248058056 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret") pod "global-pull-secret-syncer-vcf56" (UID: "913732e3-f194-491d-9b20-84dcaf859bff") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:51.887211 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.887184    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-17.ec2.internal" event="NodeReady"
Sep 11 16:17:51.887360 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.887295    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 16:17:51.924358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.924327    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:17:51.930141 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.930118    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"]
Sep 11 16:17:51.930332 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.930311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:51.933507 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.933480    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:51.933637 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.933614    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:51.933637 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.933627    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:51.933637 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.933637    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhvhb for pod openshift-network-diagnostics/network-check-target-87g4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:51.933796 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:51.933677    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb podName:3136b33e-fd4d-4743-98c0-043730b3d595 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.933665339 +0000 UTC m=+66.349148297 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhvhb" (UniqueName: "kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb") pod "network-check-target-87g4d" (UID: "3136b33e-fd4d-4743-98c0-043730b3d595") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:51.934260 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.934243    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 16:17:51.934617 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.934560    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jml27\""
Sep 11 16:17:51.934617 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.934574    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 16:17:51.935039 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.935024    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 16:17:51.938186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.938166    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"]
Sep 11 16:17:51.938298 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.938270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"
Sep 11 16:17:51.941180 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.941142    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:51.941278 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.941189    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-v52mj\""
Sep 11 16:17:51.941327 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.941293    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.941817 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.941801    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 16:17:51.948321 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.948277    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gm9tb"]
Sep 11 16:17:51.949253 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.948790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:51.953347 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.953329    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 16:17:51.953445 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.953333    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 16:17:51.953445 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.953392    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dgxn5\""
Sep 11 16:17:51.956752 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.956576    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6ff58f9bcb-zfvn2"]
Sep 11 16:17:51.956840 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.956808    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:51.962540 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.962519    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-s998b\""
Sep 11 16:17:51.962682 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.962650    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:51.963249 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.963237    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 16:17:51.963316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.963260    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 16:17:51.963361 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.963346    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.964622 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.964604    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"]
Sep 11 16:17:51.964701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.964637    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:17:51.964701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.964657    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gm9tb"]
Sep 11 16:17:51.964701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.964678    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-lzxh9"]
Sep 11 16:17:51.964846 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.964703    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:51.967112 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.967097    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 16:17:51.969745 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.969722    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 16:17:51.970635 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.970509    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.971077 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.971059    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 16:17:51.971437 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.971414    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8tgz6\""
Sep 11 16:17:51.971883 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.971322    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 16:17:51.972198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.971270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 16:17:51.972565 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.972550    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"]
Sep 11 16:17:51.972690 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.972673    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:51.976561 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.976540    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 16:17:51.977658 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.977637    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 16:17:51.977743 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.977704    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.978138 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.978123    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-88wzl\""
Sep 11 16:17:51.980774 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.980676    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"]
Sep 11 16:17:51.981075 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.981044    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:51.985901 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.985881    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 16:17:51.987329 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.987309    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 16:17:51.989064 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.989040    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 16:17:51.989164 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.989132    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 16:17:51.989524 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.989503    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:51.989524 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.989795    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vxddc\""
Sep 11 16:17:51.990268 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.990164    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.990999 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.990979    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"]
Sep 11 16:17:51.991087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.991070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:51.995196 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.995176    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 16:17:51.995615 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.995598    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:51.998381 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.998362    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dq7hw\""
Sep 11 16:17:51.998917 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:51.998893    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:52.000299 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.000273    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 16:17:52.012771 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.012707    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"]
Sep 11 16:17:52.012899 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.012817    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.015271 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.015243    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 16:17:52.015376 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.015338    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:52.015498 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.015478    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-npld8\""
Sep 11 16:17:52.015615 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.015500    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:52.015615 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.015534    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 16:17:52.028863 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.028845    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-56nbt"]
Sep 11 16:17:52.028977 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.028951    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.031436 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.031418    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 16:17:52.031577 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.031563    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wkfr5\""
Sep 11 16:17:52.031684 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.031669    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 16:17:52.032733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.032715    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 16:17:52.032809 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.032716    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 16:17:52.033996 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.033973    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.034078 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p9sn\" (UniqueName: \"kubernetes.io/projected/b1766afd-bc87-4584-9d5d-4d2a8b77711c-kube-api-access-5p9sn\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.034078 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034078 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/500f2611-8683-4f7a-a160-4d156709660b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.034078 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034078 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034073    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpxf8\" (UniqueName: \"kubernetes.io/projected/899e2f43-ccc0-4a47-ade4-b1f2ce465c30-kube-api-access-jpxf8\") pod \"volume-data-source-validator-86695b7cc7-7kz4h\" (UID: \"899e2f43-ccc0-4a47-ade4-b1f2ce465c30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"
Sep 11 16:17:52.034266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxfw8\" (UniqueName: \"kubernetes.io/projected/f935efa5-8978-4a0e-9860-18cee115cf92-kube-api-access-kxfw8\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.034266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-snapshots\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.034418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f935efa5-8978-4a0e-9860-18cee115cf92-serving-cert\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.034418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.034418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vnxr\" (UniqueName: \"kubernetes.io/projected/31c36e7b-09da-41bf-bea8-2aba716d5629-kube-api-access-8vnxr\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.034418 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034414    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-config\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-service-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034558    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-trusted-ca\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034581    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-tmp\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034606    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-default-certificate\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-stats-auth\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.034796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1766afd-bc87-4584-9d5d-4d2a8b77711c-serving-cert\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.035165 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.034809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spktn\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.047014 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.046983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"]
Sep 11 16:17:52.047108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.047033    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"]
Sep 11 16:17:52.047108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.047054    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-lzxh9"]
Sep 11 16:17:52.047108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.047056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.047108 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.047080    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"]
Sep 11 16:17:52.050062 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.050038    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 16:17:52.050172 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.050156    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 16:17:52.050222 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.050164    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkwjt\""
Sep 11 16:17:52.065093 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.065073    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vkpj2"]
Sep 11 16:17:52.065201 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.065182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"
Sep 11 16:17:52.067559 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.067542    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-96t2x\""
Sep 11 16:17:52.067654 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.067564    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:17:52.067654 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.067577    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:17:52.081092 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081072    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"]
Sep 11 16:17:52.081092 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081099    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ff58f9bcb-zfvn2"]
Sep 11 16:17:52.081257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-56nbt"]
Sep 11 16:17:52.081257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"]
Sep 11 16:17:52.081257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081159    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"]
Sep 11 16:17:52.081257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081175    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"]
Sep 11 16:17:52.081257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081190    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.081494 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.081198    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vkpj2"]
Sep 11 16:17:52.083643 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.083625    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 16:17:52.083833 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.083816    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 16:17:52.083899 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.083846    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 16:17:52.083973 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.083933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gkls5\""
Sep 11 16:17:52.135394 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.135394 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-trusted-ca\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-default-certificate\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15e4ace1-d742-46af-a8f8-515f8df58776-config-volume\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135585    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-stats-auth\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e58035b6-c5c3-434c-8a77-060704f45d0b-serving-cert\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135656    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spktn\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9vfm\" (UniqueName: \"kubernetes.io/projected/a91c33b5-2bd1-4486-84a3-0fe1facdf7b3-kube-api-access-c9vfm\") pod \"network-check-source-6f8b768455-2krvv\" (UID: \"a91c33b5-2bd1-4486-84a3-0fe1facdf7b3\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/500f2611-8683-4f7a-a160-4d156709660b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.135726    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.135732    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:17:52.135849 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.135782    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:17:52.136421 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpxf8\" (UniqueName: \"kubernetes.io/projected/899e2f43-ccc0-4a47-ade4-b1f2ce465c30-kube-api-access-jpxf8\") pod \"volume-data-source-validator-86695b7cc7-7kz4h\" (UID: \"899e2f43-ccc0-4a47-ade4-b1f2ce465c30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"
Sep 11 16:17:52.136480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136431    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxfw8\" (UniqueName: \"kubernetes.io/projected/f935efa5-8978-4a0e-9860-18cee115cf92-kube-api-access-kxfw8\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.136480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136457    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7prpc\" (UniqueName: \"kubernetes.io/projected/e58035b6-c5c3-434c-8a77-060704f45d0b-kube-api-access-7prpc\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.136589 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.136517    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.636479601 +0000 UTC m=+35.051962569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:17:52.136589 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b018bb4-0df4-4046-be03-00d7950c316c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.136710 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f935efa5-8978-4a0e-9860-18cee115cf92-serving-cert\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.136710 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.136710 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmm69\" (UniqueName: \"kubernetes.io/projected/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-kube-api-access-wmm69\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.136854 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.136764    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:17:52.136854 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.136781    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:17:52.136854 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.136768    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.636750013 +0000 UTC m=+35.052232973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:17:52.136854 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.136851    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.63683244 +0000 UTC m=+35.052315405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:17:52.137080 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/500f2611-8683-4f7a-a160-4d156709660b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.137080 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.137080 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.136930    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e4ace1-d742-46af-a8f8-515f8df58776-tmp-dir\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.137080 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-service-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.137080 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137049    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll2bw\" (UniqueName: \"kubernetes.io/projected/5b018bb4-0df4-4046-be03-00d7950c316c-kube-api-access-ll2bw\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137100    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-config\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-tmp\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17522b22-a2c6-4428-aeba-537dca1c65f5-cert\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.137319 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs5pc\" (UniqueName: \"kubernetes.io/projected/6eac51b9-5627-4d01-b33c-efb8655ab211-kube-api-access-gs5pc\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1766afd-bc87-4584-9d5d-4d2a8b77711c-serving-cert\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137379    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzwt\" (UniqueName: \"kubernetes.io/projected/15e4ace1-d742-46af-a8f8-515f8df58776-kube-api-access-qhzwt\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137457    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9sn\" (UniqueName: \"kubernetes.io/projected/b1766afd-bc87-4584-9d5d-4d2a8b77711c-kube-api-access-5p9sn\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gph4h\" (UniqueName: \"kubernetes.io/projected/17522b22-a2c6-4428-aeba-537dca1c65f5-kube-api-access-gph4h\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e58035b6-c5c3-434c-8a77-060704f45d0b-config\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eac51b9-5627-4d01-b33c-efb8655ab211-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.137608 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137598    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137642    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-snapshots\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137677    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vnxr\" (UniqueName: \"kubernetes.io/projected/31c36e7b-09da-41bf-bea8-2aba716d5629-kube-api-access-8vnxr\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eac51b9-5627-4d01-b33c-efb8655ab211-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.138046 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.137869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-trusted-ca\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.138530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.138494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-service-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.138741 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.138679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f935efa5-8978-4a0e-9860-18cee115cf92-config\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.139073    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.139316    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.639295073 +0000 UTC m=+35.054778054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.139411    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-snapshots\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.139662    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1766afd-bc87-4584-9d5d-4d2a8b77711c-tmp\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.140487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.140581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1766afd-bc87-4584-9d5d-4d2a8b77711c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.141479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f935efa5-8978-4a0e-9860-18cee115cf92-serving-cert\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.141727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-default-certificate\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.141939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1766afd-bc87-4584-9d5d-4d2a8b77711c-serving-cert\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.143066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.142371    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-stats-auth\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.144640 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.144613    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.144739 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.144675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.147447 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.147424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxfw8\" (UniqueName: \"kubernetes.io/projected/f935efa5-8978-4a0e-9860-18cee115cf92-kube-api-access-kxfw8\") pod \"console-operator-696884fb9c-gm9tb\" (UID: \"f935efa5-8978-4a0e-9860-18cee115cf92\") " pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.147678 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.147658    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.151035 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.151012    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpxf8\" (UniqueName: \"kubernetes.io/projected/899e2f43-ccc0-4a47-ade4-b1f2ce465c30-kube-api-access-jpxf8\") pod \"volume-data-source-validator-86695b7cc7-7kz4h\" (UID: \"899e2f43-ccc0-4a47-ade4-b1f2ce465c30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"
Sep 11 16:17:52.151406 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.151386    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.151521 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.151500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spktn\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.152405 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.152381    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vnxr\" (UniqueName: \"kubernetes.io/projected/31c36e7b-09da-41bf-bea8-2aba716d5629-kube-api-access-8vnxr\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.152485 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.152413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9sn\" (UniqueName: \"kubernetes.io/projected/b1766afd-bc87-4584-9d5d-4d2a8b77711c-kube-api-access-5p9sn\") pod \"insights-operator-5844b856db-lzxh9\" (UID: \"b1766afd-bc87-4584-9d5d-4d2a8b77711c\") " pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.238358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzwt\" (UniqueName: \"kubernetes.io/projected/15e4ace1-d742-46af-a8f8-515f8df58776-kube-api-access-qhzwt\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.238358 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gph4h\" (UniqueName: \"kubernetes.io/projected/17522b22-a2c6-4428-aeba-537dca1c65f5-kube-api-access-gph4h\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.238624 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e58035b6-c5c3-434c-8a77-060704f45d0b-config\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.238624 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238394    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eac51b9-5627-4d01-b33c-efb8655ab211-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.238735 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238611    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.238735 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eac51b9-5627-4d01-b33c-efb8655ab211-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.238735 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.238722    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:17:52.238883 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.238883 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.238777    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.738761991 +0000 UTC m=+35.154244949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:17:52.238883 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15e4ace1-d742-46af-a8f8-515f8df58776-config-volume\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.238883 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.238841    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:52.238883 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e58035b6-c5c3-434c-8a77-060704f45d0b-serving-cert\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.238906    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.738890443 +0000 UTC m=+35.154373408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238925    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e58035b6-c5c3-434c-8a77-060704f45d0b-config\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.238987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9vfm\" (UniqueName: \"kubernetes.io/projected/a91c33b5-2bd1-4486-84a3-0fe1facdf7b3-kube-api-access-c9vfm\") pod \"network-check-source-6f8b768455-2krvv\" (UID: \"a91c33b5-2bd1-4486-84a3-0fe1facdf7b3\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7prpc\" (UniqueName: \"kubernetes.io/projected/e58035b6-c5c3-434c-8a77-060704f45d0b-kube-api-access-7prpc\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b018bb4-0df4-4046-be03-00d7950c316c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmm69\" (UniqueName: \"kubernetes.io/projected/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-kube-api-access-wmm69\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.239178 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e4ace1-d742-46af-a8f8-515f8df58776-tmp-dir\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll2bw\" (UniqueName: \"kubernetes.io/projected/5b018bb4-0df4-4046-be03-00d7950c316c-kube-api-access-ll2bw\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239247    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eac51b9-5627-4d01-b33c-efb8655ab211-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17522b22-a2c6-4428-aeba-537dca1c65f5-cert\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239302    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs5pc\" (UniqueName: \"kubernetes.io/projected/6eac51b9-5627-4d01-b33c-efb8655ab211-kube-api-access-gs5pc\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15e4ace1-d742-46af-a8f8-515f8df58776-config-volume\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.239529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e4ace1-d742-46af-a8f8-515f8df58776-tmp-dir\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.239816 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.239570    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:52.239816 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.239617    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:52.739600875 +0000 UTC m=+35.155083837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:52.239880 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.239855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b018bb4-0df4-4046-be03-00d7950c316c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.241017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.240996    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eac51b9-5627-4d01-b33c-efb8655ab211-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.241220 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.241202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e58035b6-c5c3-434c-8a77-060704f45d0b-serving-cert\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.241596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.241580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17522b22-a2c6-4428-aeba-537dca1c65f5-cert\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.248431 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.248384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gph4h\" (UniqueName: \"kubernetes.io/projected/17522b22-a2c6-4428-aeba-537dca1c65f5-kube-api-access-gph4h\") pod \"ingress-canary-vkpj2\" (UID: \"17522b22-a2c6-4428-aeba-537dca1c65f5\") " pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.249631 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.249612    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"
Sep 11 16:17:52.249980 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.249939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs5pc\" (UniqueName: \"kubernetes.io/projected/6eac51b9-5627-4d01-b33c-efb8655ab211-kube-api-access-gs5pc\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jsz5c\" (UID: \"6eac51b9-5627-4d01-b33c-efb8655ab211\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.250294 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.250241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7prpc\" (UniqueName: \"kubernetes.io/projected/e58035b6-c5c3-434c-8a77-060704f45d0b-kube-api-access-7prpc\") pod \"service-ca-operator-cf7c776b7-cdlgj\" (UID: \"e58035b6-c5c3-434c-8a77-060704f45d0b\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.250294 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.250261    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll2bw\" (UniqueName: \"kubernetes.io/projected/5b018bb4-0df4-4046-be03-00d7950c316c-kube-api-access-ll2bw\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.250398 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.250310    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmm69\" (UniqueName: \"kubernetes.io/projected/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-kube-api-access-wmm69\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.250398 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.250363    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9vfm\" (UniqueName: \"kubernetes.io/projected/a91c33b5-2bd1-4486-84a3-0fe1facdf7b3-kube-api-access-c9vfm\") pod \"network-check-source-6f8b768455-2krvv\" (UID: \"a91c33b5-2bd1-4486-84a3-0fe1facdf7b3\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"
Sep 11 16:17:52.250547 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.250531    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzwt\" (UniqueName: \"kubernetes.io/projected/15e4ace1-d742-46af-a8f8-515f8df58776-kube-api-access-qhzwt\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.265518 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.265480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:17:52.284301 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.284285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-lzxh9"
Sep 11 16:17:52.292428 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.292411    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"
Sep 11 16:17:52.301948 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.301807    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="57038c6824ee98254394ddc22653b35d842dc08c3b52deb66d3b52785f5d780d" exitCode=0
Sep 11 16:17:52.301948 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.301877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"57038c6824ee98254394ddc22653b35d842dc08c3b52deb66d3b52785f5d780d"}
Sep 11 16:17:52.321681 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.321453    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"
Sep 11 16:17:52.373217 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.373138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"
Sep 11 16:17:52.389509 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.389445    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vkpj2"
Sep 11 16:17:52.577094 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.577066    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h"]
Sep 11 16:17:52.579807 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.579787    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gm9tb"]
Sep 11 16:17:52.582747 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:52.582723    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf935efa5_8978_4a0e_9860_18cee115cf92.slice/crio-d57ff58a0a760d7bc8b484762ba08f91658dd0f921390bc4f854d46168610a8b WatchSource:0}: Error finding container d57ff58a0a760d7bc8b484762ba08f91658dd0f921390bc4f854d46168610a8b: Status 404 returned error can't find the container with id d57ff58a0a760d7bc8b484762ba08f91658dd0f921390bc4f854d46168610a8b
Sep 11 16:17:52.582923 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:52.582900    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899e2f43_ccc0_4a47_ade4_b1f2ce465c30.slice/crio-e51f832acf4a194c9085cd7abce0faf7105022f589571363b6ad28e85ed25380 WatchSource:0}: Error finding container e51f832acf4a194c9085cd7abce0faf7105022f589571363b6ad28e85ed25380: Status 404 returned error can't find the container with id e51f832acf4a194c9085cd7abce0faf7105022f589571363b6ad28e85ed25380
Sep 11 16:17:52.611730 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.611708    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-lzxh9"]
Sep 11 16:17:52.642748 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.642724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:52.642867 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.642757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.642867 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.642784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:52.642867 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.642809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.642885    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.642889    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.642973    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.64293619 +0000 UTC m=+36.058419153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.642893    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.643003    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.642986269 +0000 UTC m=+36.058469231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.643019    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.643011142 +0000 UTC m=+36.058494105 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.643004    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:17:52.643079 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.643065    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.643055911 +0000 UTC m=+36.058538873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:17:52.647750 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.647728    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c"]
Sep 11 16:17:52.696003 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.695979    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj"]
Sep 11 16:17:52.714613 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.714576    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2krvv"]
Sep 11 16:17:52.715779 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.715724    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vkpj2"]
Sep 11 16:17:52.718987 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:17:52.718736    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91c33b5_2bd1_4486_84a3_0fe1facdf7b3.slice/crio-b161f7dfdbb4de59067080d3dfd7f597d4e4656ec4631dc427d634b19f914a4e WatchSource:0}: Error finding container b161f7dfdbb4de59067080d3dfd7f597d4e4656ec4631dc427d634b19f914a4e: Status 404 returned error can't find the container with id b161f7dfdbb4de59067080d3dfd7f597d4e4656ec4631dc427d634b19f914a4e
Sep 11 16:17:52.743944 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.743907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:52.744062 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.744031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:52.744118 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:52.744061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:52.744118 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744068    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:52.744215 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744125    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.744105963 +0000 UTC m=+36.159588923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:52.744215 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744163    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:17:52.744215 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744179    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:52.744215 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744202    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.744192322 +0000 UTC m=+36.159675281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:17:52.744355 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:52.744234    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:53.744218346 +0000 UTC m=+36.159701318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:53.105792 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.105740    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:17:53.105954 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.105863    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:17:53.105954 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.105894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:17:53.110724 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.110593    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:17:53.110724 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.110640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\""
Sep 11 16:17:53.110724 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.110697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\""
Sep 11 16:17:53.111415 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.111391    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:17:53.309932 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.309898    2566 generic.go:356] "Generic (PLEG): container finished" podID="4c64e971-ff67-44a8-959e-48fc454e6d36" containerID="4f0f9010fb7a38eeeee58f0e8d38f6e1e2b5be0ba0bc1e6f006bd100904bfdc3" exitCode=0
Sep 11 16:17:53.310706 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.310007    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerDied","Data":"4f0f9010fb7a38eeeee58f0e8d38f6e1e2b5be0ba0bc1e6f006bd100904bfdc3"}
Sep 11 16:17:53.315217 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.315132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vkpj2" event={"ID":"17522b22-a2c6-4428-aeba-537dca1c65f5","Type":"ContainerStarted","Data":"cd66a4ce79a3c15923c8957a93516a4c672589a14ba222986e4d12f3f118f82e"}
Sep 11 16:17:53.317089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.316873    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" event={"ID":"f935efa5-8978-4a0e-9860-18cee115cf92","Type":"ContainerStarted","Data":"d57ff58a0a760d7bc8b484762ba08f91658dd0f921390bc4f854d46168610a8b"}
Sep 11 16:17:53.319028 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.318973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c" event={"ID":"6eac51b9-5627-4d01-b33c-efb8655ab211","Type":"ContainerStarted","Data":"b463f7f52b5cef27f2660b24cabbad2be71afbab6c4ab99efc774510f73b92bb"}
Sep 11 16:17:53.322765 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.322719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h" event={"ID":"899e2f43-ccc0-4a47-ade4-b1f2ce465c30","Type":"ContainerStarted","Data":"e51f832acf4a194c9085cd7abce0faf7105022f589571363b6ad28e85ed25380"}
Sep 11 16:17:53.324219 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.324143    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-lzxh9" event={"ID":"b1766afd-bc87-4584-9d5d-4d2a8b77711c","Type":"ContainerStarted","Data":"fd63528a52bf164b62e9ebbdcf51df09d0080e4c60288f9341ee54bf35443962"}
Sep 11 16:17:53.326147 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.326095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv" event={"ID":"a91c33b5-2bd1-4486-84a3-0fe1facdf7b3","Type":"ContainerStarted","Data":"b161f7dfdbb4de59067080d3dfd7f597d4e4656ec4631dc427d634b19f914a4e"}
Sep 11 16:17:53.327447 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.327422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj" event={"ID":"e58035b6-c5c3-434c-8a77-060704f45d0b","Type":"ContainerStarted","Data":"a2adce0e5380ee85fc5c31f4e430de84b78c040e2545c0b4e5dc4dc724ac9b9a"}
Sep 11 16:17:53.653826 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.653737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:53.653826 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.653788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:53.654087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.653832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:53.654087 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.653865    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:53.654087 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654077    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.654057032 +0000 UTC m=+38.069539996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:17:53.654545 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654526    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:17:53.654605 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654591    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.654573086 +0000 UTC m=+38.070056050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:17:53.654666 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654652    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:17:53.654718 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654687    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.654676019 +0000 UTC m=+38.070158984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:17:53.654773 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654755    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:17:53.654773 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654766    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:17:53.654859 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.654801    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.654790742 +0000 UTC m=+38.070273714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.754938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.755024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:53.755074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755147    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755226    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.755206362 +0000 UTC m=+38.170689341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:17:53.755298 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755276    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:53.755737 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755352    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.755333002 +0000 UTC m=+38.170815976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:53.755737 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755561    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:53.755737 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:53.755620    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:55.755603391 +0000 UTC m=+38.171086354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:54.340485 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:54.339319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wznqf" event={"ID":"4c64e971-ff67-44a8-959e-48fc454e6d36","Type":"ContainerStarted","Data":"ddb1d8c1e88227ecd29a6500e75e0aba7d572c04bcd2666f552d8e2b68223ed5"}
Sep 11 16:17:55.677218 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.677172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.677231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.677273    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.677303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677379    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677421    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677442    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677445    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.677425655 +0000 UTC m=+42.092908628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677478    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677482    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.677472505 +0000 UTC m=+42.092955470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677536    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.677518592 +0000 UTC m=+42.093001554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:17:55.677648 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.677550    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.677541179 +0000 UTC m=+42.093024139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:17:55.778426 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.778377    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:55.778596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.778462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:55.778596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:55.778513    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:55.778708 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778592    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:55.778708 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778622    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:17:55.778708 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778683    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.778659989 +0000 UTC m=+42.194142948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:17:55.778708 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778704    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.778694488 +0000 UTC m=+42.194177447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:55.778902 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778746    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:55.778902 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:55.778815    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.778796754 +0000 UTC m=+42.194279731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:59.717384 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.717341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:17:59.717384 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.717389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.717419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.717440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717527    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717587    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.717574119 +0000 UTC m=+50.133057078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717609    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.717594312 +0000 UTC m=+50.133077270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717528    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717680    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.717667351 +0000 UTC m=+50.133150309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717531    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717701    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:17:59.717788 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.717735    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.717725512 +0000 UTC m=+50.133208472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:17:59.818433 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.818395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:17:59.818563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.818446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:17:59.818563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:17:59.818494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:17:59.818640 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818568    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:59.818640 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818601    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:17:59.818640 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818627    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:59.818727 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818657    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.818637791 +0000 UTC m=+50.234120772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:17:59.818727 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818675    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.818668763 +0000 UTC m=+50.234151722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:17:59.818799 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:17:59.818726    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:07.818720345 +0000 UTC m=+50.234203304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:18:00.359986 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.359937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vkpj2" event={"ID":"17522b22-a2c6-4428-aeba-537dca1c65f5","Type":"ContainerStarted","Data":"81f4a94ee1f70770b96dec8f318290087388869171fc71305dc05d6795451ab7"}
Sep 11 16:18:00.362188 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.362125    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h" event={"ID":"899e2f43-ccc0-4a47-ade4-b1f2ce465c30","Type":"ContainerStarted","Data":"4133dd2c9e6e81f315c4fa3b38bc7e63fe56a926690a920517f864aa48dc7270"}
Sep 11 16:18:00.365445 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.365415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj" event={"ID":"e58035b6-c5c3-434c-8a77-060704f45d0b","Type":"ContainerStarted","Data":"6a2bcfc4d7eca4c82a44d68da874cd8f14b058f9d34cb6815dc1cd3bdef68b5c"}
Sep 11 16:18:00.401618 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.401559    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wznqf" podStartSLOduration=14.326697081 podStartE2EDuration="42.401538713s" totalImagesPullingTime="28.074841632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:54.366082193 +0000 UTC m=+36.781565175" watchObservedRunningTime="2026-09-11 16:18:00.401538713 +0000 UTC m=+42.817021695"
Sep 11 16:18:00.402273 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.402228    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7kz4h" podStartSLOduration=3.955724436 podStartE2EDuration="11.402215694s" totalImagesPullingTime="7.446491258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:00.401405615 +0000 UTC m=+42.816888597" watchObservedRunningTime="2026-09-11 16:18:00.402215694 +0000 UTC m=+42.817698677"
Sep 11 16:18:00.425609 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:00.425402    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vkpj2" podStartSLOduration=1.982184935 podStartE2EDuration="9.42531079s" totalImagesPullingTime="7.443125855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:00.424266269 +0000 UTC m=+42.839749250" watchObservedRunningTime="2026-09-11 16:18:00.42531079 +0000 UTC m=+42.840793773"
Sep 11 16:18:01.373873 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.373842    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/0.log"
Sep 11 16:18:01.374523 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.373895    2566 generic.go:356] "Generic (PLEG): container finished" podID="f935efa5-8978-4a0e-9860-18cee115cf92" containerID="5a597e6c4228863ba797a2d02c375f2765b3d3d718a16568513387b5b65d4c8b" exitCode=255
Sep 11 16:18:01.374523 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.374082    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" event={"ID":"f935efa5-8978-4a0e-9860-18cee115cf92","Type":"ContainerDied","Data":"5a597e6c4228863ba797a2d02c375f2765b3d3d718a16568513387b5b65d4c8b"}
Sep 11 16:18:01.374621 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.374577    2566 scope.go:117] "RemoveContainer" containerID="5a597e6c4228863ba797a2d02c375f2765b3d3d718a16568513387b5b65d4c8b"
Sep 11 16:18:01.377810 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.377779    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c" event={"ID":"6eac51b9-5627-4d01-b33c-efb8655ab211","Type":"ContainerStarted","Data":"54b1f32feceb7b2cddcaf56c90d456bf5f767f9ee3ce2526d126374700705e27"}
Sep 11 16:18:01.379780 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.379748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-lzxh9" event={"ID":"b1766afd-bc87-4584-9d5d-4d2a8b77711c","Type":"ContainerStarted","Data":"814dc9e3a59741fef90074b91eb96641e7ae2427920663e4745e79c68095acae"}
Sep 11 16:18:01.381469 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.381443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv" event={"ID":"a91c33b5-2bd1-4486-84a3-0fe1facdf7b3","Type":"ContainerStarted","Data":"7bb0213cab562acc218f8345d4cb10c458605b2acd40cef8b5343ca4299a7565"}
Sep 11 16:18:01.405685 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.405658    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x58vr"]
Sep 11 16:18:01.439043 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.438509    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x58vr"]
Sep 11 16:18:01.439043 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.438629    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.445169 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.445150    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 16:18:01.447823 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.447803    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 16:18:01.448161 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.448141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ddmk9\""
Sep 11 16:18:01.454909 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.454860    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-lzxh9" podStartSLOduration=5.009877523 podStartE2EDuration="12.454841771s" totalImagesPullingTime="7.444964248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.454091213 +0000 UTC m=+43.869574193" watchObservedRunningTime="2026-09-11 16:18:01.454841771 +0000 UTC m=+43.870324754"
Sep 11 16:18:01.491504 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.491290    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c" podStartSLOduration=5.045143994 podStartE2EDuration="12.491270566s" totalImagesPullingTime="7.446126572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.491243255 +0000 UTC m=+43.906726237" watchObservedRunningTime="2026-09-11 16:18:01.491270566 +0000 UTC m=+43.906753548"
Sep 11 16:18:01.533647 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.533617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b0a960-b5f5-4c45-8547-bb1e00e0be73-data-volume\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.533785 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.533692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.533863 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.533846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrsrz\" (UniqueName: \"kubernetes.io/projected/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-api-access-mrsrz\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.534119 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.534101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b0a960-b5f5-4c45-8547-bb1e00e0be73-crio-socket\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.536416 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.536267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.538391 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.538342    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2krvv" podStartSLOduration=5.082228507 podStartE2EDuration="12.538321742s" totalImagesPullingTime="7.456093235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.511442519 +0000 UTC m=+43.926925493" watchObservedRunningTime="2026-09-11 16:18:01.538321742 +0000 UTC m=+43.953804726"
Sep 11 16:18:01.559421 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.559375    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj" podStartSLOduration=5.116529988 podStartE2EDuration="12.559359339s" totalImagesPullingTime="7.442829351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.538769303 +0000 UTC m=+43.954252285" watchObservedRunningTime="2026-09-11 16:18:01.559359339 +0000 UTC m=+43.974842625"
Sep 11 16:18:01.636987 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.636891    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b0a960-b5f5-4c45-8547-bb1e00e0be73-crio-socket\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637135 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.636985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637135 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.637113    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b0a960-b5f5-4c45-8547-bb1e00e0be73-crio-socket\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637237 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:01.637142    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:01.637237 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:01.637204    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.137187086 +0000 UTC m=+44.552670049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:01.637237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.637148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b0a960-b5f5-4c45-8547-bb1e00e0be73-data-volume\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637487 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.637246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637487 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.637318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrsrz\" (UniqueName: \"kubernetes.io/projected/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-api-access-mrsrz\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.637487 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:01.637377    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:01.637487 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:01.637449    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.137421434 +0000 UTC m=+44.552904414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:01.637487 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.637475    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b0a960-b5f5-4c45-8547-bb1e00e0be73-data-volume\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:01.648577 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:01.648551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrsrz\" (UniqueName: \"kubernetes.io/projected/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-api-access-mrsrz\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:02.143153 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.143120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:02.143399 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.143226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:02.143399 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:02.143272    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:02.143399 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:02.143339    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.143323705 +0000 UTC m=+45.558806663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:02.143399 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:02.143340    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:02.143399 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:02.143394    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.143377771 +0000 UTC m=+45.558860750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:02.266060 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.266026    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:02.266216 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.266083    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:02.386551 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.386524    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:18:02.387017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.387002    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/0.log"
Sep 11 16:18:02.387079 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.387036    2566 generic.go:356] "Generic (PLEG): container finished" podID="f935efa5-8978-4a0e-9860-18cee115cf92" containerID="776c93927c34e06d1f68ca9b37ff86a492309c2173ecf6a7a10cd0f42ad0f6d7" exitCode=255
Sep 11 16:18:02.387127 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.387113    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" event={"ID":"f935efa5-8978-4a0e-9860-18cee115cf92","Type":"ContainerDied","Data":"776c93927c34e06d1f68ca9b37ff86a492309c2173ecf6a7a10cd0f42ad0f6d7"}
Sep 11 16:18:02.387194 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.387182    2566 scope.go:117] "RemoveContainer" containerID="5a597e6c4228863ba797a2d02c375f2765b3d3d718a16568513387b5b65d4c8b"
Sep 11 16:18:02.387389 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:02.387371    2566 scope.go:117] "RemoveContainer" containerID="776c93927c34e06d1f68ca9b37ff86a492309c2173ecf6a7a10cd0f42ad0f6d7"
Sep 11 16:18:02.387587 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:02.387568    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gm9tb_openshift-console-operator(f935efa5-8978-4a0e-9860-18cee115cf92)\"" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" podUID="f935efa5-8978-4a0e-9860-18cee115cf92"
Sep 11 16:18:03.150785 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:03.150752    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:03.151010 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:03.150826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:03.151010 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:03.150893    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:03.151010 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:03.150929    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:03.151010 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:03.150988    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.15095239 +0000 UTC m=+47.566435368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:03.151185 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:03.151015    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.151003893 +0000 UTC m=+47.566486855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:03.392430 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:03.392405    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:18:03.392748 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:03.392693    2566 scope.go:117] "RemoveContainer" containerID="776c93927c34e06d1f68ca9b37ff86a492309c2173ecf6a7a10cd0f42ad0f6d7"
Sep 11 16:18:03.392841 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:03.392823    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gm9tb_openshift-console-operator(f935efa5-8978-4a0e-9860-18cee115cf92)\"" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" podUID="f935efa5-8978-4a0e-9860-18cee115cf92"
Sep 11 16:18:03.553614 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:03.553569    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rbxbl_4b554964-07b2-4935-9234-e36967e235d7/dns-node-resolver/0.log"
Sep 11 16:18:04.353553 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:04.353526    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97z2j_9ce9172b-8309-4a61-980b-f98dbcdf1da4/node-ca/0.log"
Sep 11 16:18:05.168015 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:05.167981    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:05.168397 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:05.168055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:05.168397 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:05.168124    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:05.168397 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:05.168126    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:05.168397 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:05.168177    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.168163409 +0000 UTC m=+51.583646368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:05.168397 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:05.168191    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.168184144 +0000 UTC m=+51.583667103 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:05.354034 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:05.354006    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vkpj2_17522b22-a2c6-4428-aeba-537dca1c65f5/serve-healthcheck-canary/0.log"
Sep 11 16:18:07.792235 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.792191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:18:07.792235 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.792238    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.792306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.792334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792343    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792406    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert podName:500f2611-8683-4f7a-a160-4d156709660b nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.79238813 +0000 UTC m=+66.207871111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-s2r4s" (UID: "500f2611-8683-4f7a-a160-4d156709660b") : secret "networking-console-plugin-cert" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792427    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792433    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792503    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86767db879-d6gml: secret "image-registry-tls" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792438    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.792428172 +0000 UTC m=+66.207911141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : configmap references non-existent config key: service-ca.crt
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792536    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs podName:31c36e7b-09da-41bf-bea8-2aba716d5629 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.792519325 +0000 UTC m=+66.208002288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs") pod "router-default-6ff58f9bcb-zfvn2" (UID: "31c36e7b-09da-41bf-bea8-2aba716d5629") : secret "router-metrics-certs-default" not found
Sep 11 16:18:07.792671 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.792559    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls podName:1bcb1516-9c13-4691-9f3f-aa1e13b51ed0 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.792549447 +0000 UTC m=+66.208032416 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls") pod "image-registry-86767db879-d6gml" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0") : secret "image-registry-tls" not found
Sep 11 16:18:07.893036 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.893008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:18:07.893169 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.893050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:18:07.893169 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893138    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 16:18:07.893251 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893187    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:18:07.893251 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893192    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls podName:5b018bb4-0df4-4046-be03-00d7950c316c nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.893178463 +0000 UTC m=+66.308661426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-shcnn" (UID: "5b018bb4-0df4-4046-be03-00d7950c316c") : secret "cluster-monitoring-operator-tls" not found
Sep 11 16:18:07.893251 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893229    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls podName:719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.893219121 +0000 UTC m=+66.308702080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2fbhd" (UID: "719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a") : secret "samples-operator-tls" not found
Sep 11 16:18:07.893251 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:07.893240    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:07.893403 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893325    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:18:07.893403 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:07.893345    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls podName:15e4ace1-d742-46af-a8f8-515f8df58776 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:23.893338343 +0000 UTC m=+66.308821300 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls") pod "dns-default-56nbt" (UID: "15e4ace1-d742-46af-a8f8-515f8df58776") : secret "dns-default-metrics-tls" not found
Sep 11 16:18:09.202622 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:09.202578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:09.203006 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:09.202665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:09.203006 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:09.202730    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:09.203006 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:09.202799    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls podName:29b0a960-b5f5-4c45-8547-bb1e00e0be73 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:17.202783426 +0000 UTC m=+59.618266389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x58vr" (UID: "29b0a960-b5f5-4c45-8547-bb1e00e0be73") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:09.203303 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:09.203286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b0a960-b5f5-4c45-8547-bb1e00e0be73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:12.265631 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.265601    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:12.266057 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.265652    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:12.266057 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.265933    2566 scope.go:117] "RemoveContainer" containerID="776c93927c34e06d1f68ca9b37ff86a492309c2173ecf6a7a10cd0f42ad0f6d7"
Sep 11 16:18:12.421531 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.421508    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:18:12.421672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.421552    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" event={"ID":"f935efa5-8978-4a0e-9860-18cee115cf92","Type":"ContainerStarted","Data":"c6e0b1f4d51d181204d169666e414cf1ec05c02be2266fc507be0aae6b3022a1"}
Sep 11 16:18:12.421854 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.421826    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:12.441056 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:12.441012    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb" podStartSLOduration=15.855001601 podStartE2EDuration="23.440997269s" totalImagesPullingTime="7.585995668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:12.440219476 +0000 UTC m=+54.855702470" watchObservedRunningTime="2026-09-11 16:18:12.440997269 +0000 UTC m=+54.856480250"
Sep 11 16:18:13.409564 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:13.409537    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-gm9tb"
Sep 11 16:18:17.276310 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:17.272671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:17.276310 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:17.275665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b0a960-b5f5-4c45-8547-bb1e00e0be73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x58vr\" (UID: \"29b0a960-b5f5-4c45-8547-bb1e00e0be73\") " pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:17.294781 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:17.294759    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh8vf"
Sep 11 16:18:17.349142 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:17.349113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x58vr"
Sep 11 16:18:17.623058 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:17.623012    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x58vr"]
Sep 11 16:18:17.625306 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:17.625284    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b0a960_b5f5_4c45_8547_bb1e00e0be73.slice/crio-35657864c9b43249c57ef07d60851e8b745f35322623a1f40948e80b1976d03a WatchSource:0}: Error finding container 35657864c9b43249c57ef07d60851e8b745f35322623a1f40948e80b1976d03a: Status 404 returned error can't find the container with id 35657864c9b43249c57ef07d60851e8b745f35322623a1f40948e80b1976d03a
Sep 11 16:18:18.443288 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:18.443255    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x58vr" event={"ID":"29b0a960-b5f5-4c45-8547-bb1e00e0be73","Type":"ContainerStarted","Data":"d3a4ed19a5865f259afbecd31d756d7725621528bd03e2cbf562eae0e5e3f463"}
Sep 11 16:18:18.443578 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:18.443316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x58vr" event={"ID":"29b0a960-b5f5-4c45-8547-bb1e00e0be73","Type":"ContainerStarted","Data":"05e96af22da494be2125b46964d9b9c5496bd379647a89c32cf6e04317f9b2cc"}
Sep 11 16:18:18.443578 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:18.443336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x58vr" event={"ID":"29b0a960-b5f5-4c45-8547-bb1e00e0be73","Type":"ContainerStarted","Data":"35657864c9b43249c57ef07d60851e8b745f35322623a1f40948e80b1976d03a"}
Sep 11 16:18:20.454389 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:20.454357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x58vr" event={"ID":"29b0a960-b5f5-4c45-8547-bb1e00e0be73","Type":"ContainerStarted","Data":"50b00c69c3b7a1142ce264a7ef2a2b9dd4ed0866749956833834f091b0f8f09f"}
Sep 11 16:18:20.476148 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:20.476107    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x58vr" podStartSLOduration=17.518857907 podStartE2EDuration="19.476092338s" totalImagesPullingTime="1.957234431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:20.47440045 +0000 UTC m=+62.889883442" watchObservedRunningTime="2026-09-11 16:18:20.476092338 +0000 UTC m=+62.891575319"
Sep 11 16:18:23.830287 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.830244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:23.830287 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.830293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:23.830776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.830412    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:18:23.830776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.830443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:18:23.830776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.830470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:23.831099 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.831074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31c36e7b-09da-41bf-bea8-2aba716d5629-service-ca-bundle\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:23.832834 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.832805    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/500f2611-8683-4f7a-a160-4d156709660b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-s2r4s\" (UID: \"500f2611-8683-4f7a-a160-4d156709660b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:18:23.833024 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.832835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c36e7b-09da-41bf-bea8-2aba716d5629-metrics-certs\") pod \"router-default-6ff58f9bcb-zfvn2\" (UID: \"31c36e7b-09da-41bf-bea8-2aba716d5629\") " pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:23.833024 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.832880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"image-registry-86767db879-d6gml\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") " pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:23.833311 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.833295    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:18:23.843335 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.843317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57db61b1-242f-476a-b2be-d0636b8328b5-metrics-certs\") pod \"network-metrics-daemon-dmr9d\" (UID: \"57db61b1-242f-476a-b2be-d0636b8328b5\") " pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:18:23.931017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.930953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:18:23.931205 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.931032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:18:23.931205 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.931064    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:23.931205 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.931084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:18:23.933387 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.933360    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e4ace1-d742-46af-a8f8-515f8df58776-metrics-tls\") pod \"dns-default-56nbt\" (UID: \"15e4ace1-d742-46af-a8f8-515f8df58776\") " pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:23.933481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.933404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b018bb4-0df4-4046-be03-00d7950c316c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-shcnn\" (UID: \"5b018bb4-0df4-4046-be03-00d7950c316c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:18:23.933540 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.933521    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2fbhd\" (UID: \"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:18:23.934249 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.934227    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:18:23.943254 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:23.943234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/913732e3-f194-491d-9b20-84dcaf859bff-original-pull-secret\") pod \"global-pull-secret-syncer-vcf56\" (UID: \"913732e3-f194-491d-9b20-84dcaf859bff\") " pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:18:24.022264 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.022228    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcf56"
Sep 11 16:18:24.032140 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.032117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:18:24.034437 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.034413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhvhb\" (UniqueName: \"kubernetes.io/projected/3136b33e-fd4d-4743-98c0-043730b3d595-kube-api-access-xhvhb\") pod \"network-check-target-87g4d\" (UID: \"3136b33e-fd4d-4743-98c0-043730b3d595\") " pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:18:24.039518 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.039494    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\""
Sep 11 16:18:24.041712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.041693    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jml27\""
Sep 11 16:18:24.047697 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.047676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dmr9d"
Sep 11 16:18:24.049555 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.049536    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:24.061028 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.061007    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dgxn5\""
Sep 11 16:18:24.069790 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.069084    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"
Sep 11 16:18:24.078204 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.078182    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8tgz6\""
Sep 11 16:18:24.086263 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.086023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:24.109092 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.109060    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dq7hw\""
Sep 11 16:18:24.116951 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.115580    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"
Sep 11 16:18:24.139853 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.139633    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wkfr5\""
Sep 11 16:18:24.148093 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.147913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"
Sep 11 16:18:24.158353 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.158327    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkwjt\""
Sep 11 16:18:24.168751 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.168544    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:24.291004 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.290922    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcf56"]
Sep 11 16:18:24.293571 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.293542    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913732e3_f194_491d_9b20_84dcaf859bff.slice/crio-b6120e97c1bcb55fb313319ba9732b81aaba985bc7725f1ef7f9c58ec7518402 WatchSource:0}: Error finding container b6120e97c1bcb55fb313319ba9732b81aaba985bc7725f1ef7f9c58ec7518402: Status 404 returned error can't find the container with id b6120e97c1bcb55fb313319ba9732b81aaba985bc7725f1ef7f9c58ec7518402
Sep 11 16:18:24.332807 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.332725    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\""
Sep 11 16:18:24.339682 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.339655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dmr9d"]
Sep 11 16:18:24.341336 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.341323    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:18:24.341970 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.341939    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57db61b1_242f_476a_b2be_d0636b8328b5.slice/crio-212d73daff9b038621bb1fb58ebd7a09a23b207a3c0089cd19af6fb46483f0f0 WatchSource:0}: Error finding container 212d73daff9b038621bb1fb58ebd7a09a23b207a3c0089cd19af6fb46483f0f0: Status 404 returned error can't find the container with id 212d73daff9b038621bb1fb58ebd7a09a23b207a3c0089cd19af6fb46483f0f0
Sep 11 16:18:24.345333 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.345305    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:18:24.350396 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.350375    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bcb1516_9c13_4691_9f3f_aa1e13b51ed0.slice/crio-31424be40847d58753b2f7ca5f5bccc70415ef3ebcf4ab63406df4f98764a7d7 WatchSource:0}: Error finding container 31424be40847d58753b2f7ca5f5bccc70415ef3ebcf4ab63406df4f98764a7d7: Status 404 returned error can't find the container with id 31424be40847d58753b2f7ca5f5bccc70415ef3ebcf4ab63406df4f98764a7d7
Sep 11 16:18:24.366012 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.365983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s"]
Sep 11 16:18:24.369477 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.369446    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500f2611_8683_4f7a_a160_4d156709660b.slice/crio-361e0ed7673627a00d3689c608db8b832e27eae38db66c4d5713d6f8152baf8b WatchSource:0}: Error finding container 361e0ed7673627a00d3689c608db8b832e27eae38db66c4d5713d6f8152baf8b: Status 404 returned error can't find the container with id 361e0ed7673627a00d3689c608db8b832e27eae38db66c4d5713d6f8152baf8b
Sep 11 16:18:24.390102 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.390076    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ff58f9bcb-zfvn2"]
Sep 11 16:18:24.392666 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.392624    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c36e7b_09da_41bf_bea8_2aba716d5629.slice/crio-8ed43b2397fa4baf5c2d9813ee584f4eaaeea227f77b5cb1fd7f6fdb8cce6f44 WatchSource:0}: Error finding container 8ed43b2397fa4baf5c2d9813ee584f4eaaeea227f77b5cb1fd7f6fdb8cce6f44: Status 404 returned error can't find the container with id 8ed43b2397fa4baf5c2d9813ee584f4eaaeea227f77b5cb1fd7f6fdb8cce6f44
Sep 11 16:18:24.412593 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.412536    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd"]
Sep 11 16:18:24.435927 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.435710    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn"]
Sep 11 16:18:24.441719 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.441689    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b018bb4_0df4_4046_be03_00d7950c316c.slice/crio-2747554502d21e39906c10ae33355e9093456a3319dc18ad37e7ca67ab71476a WatchSource:0}: Error finding container 2747554502d21e39906c10ae33355e9093456a3319dc18ad37e7ca67ab71476a: Status 404 returned error can't find the container with id 2747554502d21e39906c10ae33355e9093456a3319dc18ad37e7ca67ab71476a
Sep 11 16:18:24.459699 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.459330    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-56nbt"]
Sep 11 16:18:24.462524 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.462493    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e4ace1_d742_46af_a8f8_515f8df58776.slice/crio-77e4c2915087cd2d5133446183a874a420595991a8e408d6b62605d2665d6ff4 WatchSource:0}: Error finding container 77e4c2915087cd2d5133446183a874a420595991a8e408d6b62605d2665d6ff4: Status 404 returned error can't find the container with id 77e4c2915087cd2d5133446183a874a420595991a8e408d6b62605d2665d6ff4
Sep 11 16:18:24.473107 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.473079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dmr9d" event={"ID":"57db61b1-242f-476a-b2be-d0636b8328b5","Type":"ContainerStarted","Data":"212d73daff9b038621bb1fb58ebd7a09a23b207a3c0089cd19af6fb46483f0f0"}
Sep 11 16:18:24.474396 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.474369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-56nbt" event={"ID":"15e4ace1-d742-46af-a8f8-515f8df58776","Type":"ContainerStarted","Data":"77e4c2915087cd2d5133446183a874a420595991a8e408d6b62605d2665d6ff4"}
Sep 11 16:18:24.476288 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.476256    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcf56" event={"ID":"913732e3-f194-491d-9b20-84dcaf859bff","Type":"ContainerStarted","Data":"b6120e97c1bcb55fb313319ba9732b81aaba985bc7725f1ef7f9c58ec7518402"}
Sep 11 16:18:24.477563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.477541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn" event={"ID":"5b018bb4-0df4-4046-be03-00d7950c316c","Type":"ContainerStarted","Data":"2747554502d21e39906c10ae33355e9093456a3319dc18ad37e7ca67ab71476a"}
Sep 11 16:18:24.478949 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.478928    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s" event={"ID":"500f2611-8683-4f7a-a160-4d156709660b","Type":"ContainerStarted","Data":"361e0ed7673627a00d3689c608db8b832e27eae38db66c4d5713d6f8152baf8b"}
Sep 11 16:18:24.480481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.480455    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd" event={"ID":"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a","Type":"ContainerStarted","Data":"0ceedf42c71a08d4eb46385fde45ca7b8af294c5afa49f8e4dbb6907e8bbcafc"}
Sep 11 16:18:24.482387 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.482227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2" event={"ID":"31c36e7b-09da-41bf-bea8-2aba716d5629","Type":"ContainerStarted","Data":"8ed43b2397fa4baf5c2d9813ee584f4eaaeea227f77b5cb1fd7f6fdb8cce6f44"}
Sep 11 16:18:24.484245 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.484220    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86767db879-d6gml" event={"ID":"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0","Type":"ContainerStarted","Data":"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"}
Sep 11 16:18:24.484342 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.484264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86767db879-d6gml" event={"ID":"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0","Type":"ContainerStarted","Data":"31424be40847d58753b2f7ca5f5bccc70415ef3ebcf4ab63406df4f98764a7d7"}
Sep 11 16:18:24.484653 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.484637    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:24.507383 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.507332    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2" podStartSLOduration=35.50731452 podStartE2EDuration="35.50731452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:24.506980325 +0000 UTC m=+66.922463319" watchObservedRunningTime="2026-09-11 16:18:24.50731452 +0000 UTC m=+66.922797500"
Sep 11 16:18:24.528111 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.528047    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86767db879-d6gml" podStartSLOduration=66.528022809 podStartE2EDuration="1m6.528022809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:24.52768459 +0000 UTC m=+66.943167575" watchObservedRunningTime="2026-09-11 16:18:24.528022809 +0000 UTC m=+66.943505791"
Sep 11 16:18:24.624306 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:24.624216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-87g4d"]
Sep 11 16:18:24.627214 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:24.627180    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3136b33e_fd4d_4743_98c0_043730b3d595.slice/crio-d58bbfe13b06c9c3d513fabc2ee410990717ea1e41b11f224c6801d6d5a24591 WatchSource:0}: Error finding container d58bbfe13b06c9c3d513fabc2ee410990717ea1e41b11f224c6801d6d5a24591: Status 404 returned error can't find the container with id d58bbfe13b06c9c3d513fabc2ee410990717ea1e41b11f224c6801d6d5a24591
Sep 11 16:18:25.087041 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.086996    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:25.090134 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.090104    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:25.495355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.495225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2" event={"ID":"31c36e7b-09da-41bf-bea8-2aba716d5629","Type":"ContainerStarted","Data":"a0ec48660e4498b7b9d8a213ce4a51578c68a37acf6e92acf0a6ee13a2ca5ace"}
Sep 11 16:18:25.495585 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.495541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:25.497029 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.496954    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6ff58f9bcb-zfvn2"
Sep 11 16:18:25.499361 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.499336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-87g4d" event={"ID":"3136b33e-fd4d-4743-98c0-043730b3d595","Type":"ContainerStarted","Data":"51d5a21d6c3d799a6f2d449189b1763149e339347a510b8378495cb197bb62d9"}
Sep 11 16:18:25.499451 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.499380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-87g4d" event={"ID":"3136b33e-fd4d-4743-98c0-043730b3d595","Type":"ContainerStarted","Data":"d58bbfe13b06c9c3d513fabc2ee410990717ea1e41b11f224c6801d6d5a24591"}
Sep 11 16:18:25.499451 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.499407    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:18:25.537946 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:25.537281    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-87g4d" podStartSLOduration=67.53725861 podStartE2EDuration="1m7.53725861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:25.536355786 +0000 UTC m=+67.951838792" watchObservedRunningTime="2026-09-11 16:18:25.53725861 +0000 UTC m=+67.952741592"
Sep 11 16:18:30.321989 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.321939    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"]
Sep 11 16:18:30.344710 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.344681    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"]
Sep 11 16:18:30.344844 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.344781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:30.347574 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.347549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 16:18:30.348832 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.348807    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2b2vn\""
Sep 11 16:18:30.495466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.495428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qb5j7\" (UID: \"18a30c8f-24dc-4c49-ac72-60a469a43b5c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:30.521044 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.521007    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dmr9d" event={"ID":"57db61b1-242f-476a-b2be-d0636b8328b5","Type":"ContainerStarted","Data":"1abf105a54e1436dd19726f43487382ba2e7987aff423c167540a40c06468688"}
Sep 11 16:18:30.521221 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.521071    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dmr9d" event={"ID":"57db61b1-242f-476a-b2be-d0636b8328b5","Type":"ContainerStarted","Data":"d7907d4eeca634dbcab5ba6a6eec9a9ff33a01619a23191169d13d3bb1012e00"}
Sep 11 16:18:30.522933 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.522906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-56nbt" event={"ID":"15e4ace1-d742-46af-a8f8-515f8df58776","Type":"ContainerStarted","Data":"4fe3517f18d4e1e7ba7476001c78fdc1cd0923220f532b13705a9056a2ce36c1"}
Sep 11 16:18:30.523094 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.522952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-56nbt" event={"ID":"15e4ace1-d742-46af-a8f8-515f8df58776","Type":"ContainerStarted","Data":"1d5cac6d7fa97ba670a6f7cf847827f2c06c86d6a7a22ace7bfb8ef0925e20ab"}
Sep 11 16:18:30.523094 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.523030    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:30.524570 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.524547    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcf56" event={"ID":"913732e3-f194-491d-9b20-84dcaf859bff","Type":"ContainerStarted","Data":"5a14e2532da2383bb95d4e5e01477e3826dd43026e696953974e8e137da388ff"}
Sep 11 16:18:30.526118 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.526098    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn" event={"ID":"5b018bb4-0df4-4046-be03-00d7950c316c","Type":"ContainerStarted","Data":"2cc22619d83dc0a8a3292a19801fed05f52826d6a88c2acbcc5d29aa7b6a79f5"}
Sep 11 16:18:30.527600 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.527577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s" event={"ID":"500f2611-8683-4f7a-a160-4d156709660b","Type":"ContainerStarted","Data":"78b5ed8280a221e9d75ad1094980b6429b520acda854e2438a8b58faf8258fc6"}
Sep 11 16:18:30.529209 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.529189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd" event={"ID":"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a","Type":"ContainerStarted","Data":"03b1c705fa2af3050a1325d25fcbce974c47968fdae18be7de8885a1dad1f23d"}
Sep 11 16:18:30.529286 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.529230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd" event={"ID":"719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a","Type":"ContainerStarted","Data":"f6ae480a25d7f1e7b313e3d341ad5ea9c20e4f7af942a7ef2f73d9f963dd71c2"}
Sep 11 16:18:30.539633 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.539594    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dmr9d" podStartSLOduration=67.253851807 podStartE2EDuration="1m12.539580522s" totalImagesPullingTime="5.285728715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.538904095 +0000 UTC m=+72.954387081" watchObservedRunningTime="2026-09-11 16:18:30.539580522 +0000 UTC m=+72.955063502"
Sep 11 16:18:30.561955 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.561907    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2fbhd" podStartSLOduration=36.395172398 podStartE2EDuration="41.561891513s" totalImagesPullingTime="5.166719115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.561332781 +0000 UTC m=+72.976815761" watchObservedRunningTime="2026-09-11 16:18:30.561891513 +0000 UTC m=+72.977374546"
Sep 11 16:18:30.578183 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.578088    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcf56" podStartSLOduration=67.228087011 podStartE2EDuration="1m12.578073717s" totalImagesPullingTime="5.349986706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.576814277 +0000 UTC m=+72.992297257" watchObservedRunningTime="2026-09-11 16:18:30.578073717 +0000 UTC m=+72.993556697"
Sep 11 16:18:30.594491 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.594443    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-s2r4s" podStartSLOduration=36.336276967 podStartE2EDuration="41.594429517s" totalImagesPullingTime="5.25815255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.593378368 +0000 UTC m=+73.008861349" watchObservedRunningTime="2026-09-11 16:18:30.594429517 +0000 UTC m=+73.009912497"
Sep 11 16:18:30.596082 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.596053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qb5j7\" (UID: \"18a30c8f-24dc-4c49-ac72-60a469a43b5c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:30.596798 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:30.596501    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 16:18:30.596798 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:30.596590    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates podName:18a30c8f-24dc-4c49-ac72-60a469a43b5c nodeName:}" failed. No retries permitted until 2026-09-11 16:18:31.096567102 +0000 UTC m=+73.512050072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-qb5j7" (UID: "18a30c8f-24dc-4c49-ac72-60a469a43b5c") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 16:18:30.618069 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.618012    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-shcnn" podStartSLOduration=36.430262082 podStartE2EDuration="41.617993947s" totalImagesPullingTime="5.187731865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.616572026 +0000 UTC m=+73.032055018" watchObservedRunningTime="2026-09-11 16:18:30.617993947 +0000 UTC m=+73.033476936"
Sep 11 16:18:30.639654 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:30.639578    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-56nbt" podStartSLOduration=34.471938346 podStartE2EDuration="39.639554279s" totalImagesPullingTime="5.167615933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:30.637910254 +0000 UTC m=+73.053393248" watchObservedRunningTime="2026-09-11 16:18:30.639554279 +0000 UTC m=+73.055037261"
Sep 11 16:18:31.100613 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:31.100578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qb5j7\" (UID: \"18a30c8f-24dc-4c49-ac72-60a469a43b5c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:31.103066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:31.103042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/18a30c8f-24dc-4c49-ac72-60a469a43b5c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qb5j7\" (UID: \"18a30c8f-24dc-4c49-ac72-60a469a43b5c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:31.254269 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:31.254232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:31.563648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:31.563622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"]
Sep 11 16:18:31.565938 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:31.565914    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a30c8f_24dc_4c49_ac72_60a469a43b5c.slice/crio-05d95486f41005ea6b65a532a125749c6735c8ef1327dbb5c4e748cb0d2f79e5 WatchSource:0}: Error finding container 05d95486f41005ea6b65a532a125749c6735c8ef1327dbb5c4e748cb0d2f79e5: Status 404 returned error can't find the container with id 05d95486f41005ea6b65a532a125749c6735c8ef1327dbb5c4e748cb0d2f79e5
Sep 11 16:18:32.538828 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:32.538784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7" event={"ID":"18a30c8f-24dc-4c49-ac72-60a469a43b5c","Type":"ContainerStarted","Data":"05d95486f41005ea6b65a532a125749c6735c8ef1327dbb5c4e748cb0d2f79e5"}
Sep 11 16:18:33.544504 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:33.544467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7" event={"ID":"18a30c8f-24dc-4c49-ac72-60a469a43b5c","Type":"ContainerStarted","Data":"f2f7313bc3042e63841564a49a3954f62a74c557edc23becadef5d1598e2729f"}
Sep 11 16:18:33.544908 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:33.544650    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:33.549465 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:33.549436    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7"
Sep 11 16:18:33.562599 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:33.562544    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qb5j7" podStartSLOduration=2.093522642 podStartE2EDuration="3.562527732s" totalImagesPullingTime="1.46900509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:33.561785113 +0000 UTC m=+75.977268095" watchObservedRunningTime="2026-09-11 16:18:33.562527732 +0000 UTC m=+75.978010712"
Sep 11 16:18:40.535668 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:40.535551    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-56nbt"
Sep 11 16:18:45.503149 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:45.503122    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:18:48.208353 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:48.208320    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:18:49.806633 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.806598    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"]
Sep 11 16:18:49.809521 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.809505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:49.812341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.812319    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 16:18:49.812797 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.812778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 16:18:49.813062 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.813045    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ll66n\""
Sep 11 16:18:49.819239 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.819221    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 16:18:49.824510 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.824491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"]
Sep 11 16:18:49.851977 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.851937    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n2cj5"]
Sep 11 16:18:49.856153 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.856116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:49.859550 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.859531    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 16:18:49.859550 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.859539    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wc8kf\""
Sep 11 16:18:49.859720 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.859559    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 16:18:49.859720 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.859587    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 16:18:49.942032 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.942005    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:49.942174 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.942048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:49.942174 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.942078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5663307d-b526-4545-a021-888486bd380d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:49.942174 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:49.942102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48755\" (UniqueName: \"kubernetes.io/projected/5663307d-b526-4545-a021-888486bd380d-kube-api-access-48755\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.043494 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043494 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043496    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-root\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-wtmp\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.043686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043661    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-sys\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-textfile\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043707    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s76b9\" (UniqueName: \"kubernetes.io/projected/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-kube-api-access-s76b9\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5663307d-b526-4545-a021-888486bd380d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.043855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48755\" (UniqueName: \"kubernetes.io/projected/5663307d-b526-4545-a021-888486bd380d-kube-api-access-48755\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.043855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-metrics-client-ca\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.043855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.044069 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.044069 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.043906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.044069 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:50.044044    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 16:18:50.044204 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:50.044121    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls podName:5663307d-b526-4545-a021-888486bd380d nodeName:}" failed. No retries permitted until 2026-09-11 16:18:50.544101455 +0000 UTC m=+92.959584424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-sgdb2" (UID: "5663307d-b526-4545-a021-888486bd380d") : secret "openshift-state-metrics-tls" not found
Sep 11 16:18:50.044414 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.044397    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5663307d-b526-4545-a021-888486bd380d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.045999 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.045982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.052747 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.052727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48755\" (UniqueName: \"kubernetes.io/projected/5663307d-b526-4545-a021-888486bd380d-kube-api-access-48755\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.145173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-root\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-wtmp\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-sys\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-textfile\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s76b9\" (UniqueName: \"kubernetes.io/projected/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-kube-api-access-s76b9\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145243    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-sys\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-metrics-client-ca\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145216    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-root\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145298    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-wtmp\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145911 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:50.145563    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 16:18:50.145911 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145605    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-textfile\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.145911 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:18:50.145642    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls podName:1cd40cd2-32ce-453a-bff5-639d8f9c7ed1 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:50.645619887 +0000 UTC m=+93.061102848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls") pod "node-exporter-n2cj5" (UID: "1cd40cd2-32ce-453a-bff5-639d8f9c7ed1") : secret "node-exporter-tls" not found
Sep 11 16:18:50.146071 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-metrics-client-ca\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.146071 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.145955    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.147355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.147337    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.157401 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.157375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s76b9\" (UniqueName: \"kubernetes.io/projected/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-kube-api-access-s76b9\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.550213 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.550183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.552648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.552617    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5663307d-b526-4545-a021-888486bd380d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sgdb2\" (UID: \"5663307d-b526-4545-a021-888486bd380d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.651331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.651296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.653603 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.653577    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1cd40cd2-32ce-453a-bff5-639d8f9c7ed1-node-exporter-tls\") pod \"node-exporter-n2cj5\" (UID: \"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1\") " pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.718731 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.718693    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"
Sep 11 16:18:50.765530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.765495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n2cj5"
Sep 11 16:18:50.879636 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.879561    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:18:50.882575 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.882558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:50.888316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 16:18:50.888445 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888294    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 16:18:50.888567 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888546    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 16:18:50.888623 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888558    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 16:18:50.888810 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888791    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 16:18:50.888930 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888857    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 16:18:50.888930 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.888863    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 16:18:50.889469 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.889406    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qt8bn\""
Sep 11 16:18:50.889535 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.889517    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 16:18:50.889535 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.889524    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 16:18:50.901582 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:50.901558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:18:51.030412 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.030388    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2"]
Sep 11 16:18:51.032201 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:51.032178    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5663307d_b526_4545_a021_888486bd380d.slice/crio-be24bb156d03c63e8258ec35d7592847ead45c871b70c7c95774004293c37c69 WatchSource:0}: Error finding container be24bb156d03c63e8258ec35d7592847ead45c871b70c7c95774004293c37c69: Status 404 returned error can't find the container with id be24bb156d03c63e8258ec35d7592847ead45c871b70c7c95774004293c37c69
Sep 11 16:18:51.054690 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.054658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.054804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.054700    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.054804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.054727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.054804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.054752    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.054804 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.054780    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055051 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055156 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055226 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055208    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-config-volume\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055271 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-config-out\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055306 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-web-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055347 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055320    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw2mb\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-kube-api-access-bw2mb\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055385 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.055427 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.055415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156072 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-config-volume\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156302    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-config-out\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-web-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw2mb\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-kube-api-access-bw2mb\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156386    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156728 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.156728 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.156465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.157088 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.157062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.158617 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.158217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5d1545-9219-479d-9130-98f45e9b6b76-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.159113 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.159068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.159204 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.159174    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.160247 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.160218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.160847 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.160805    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.161035 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.161011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.161190 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.161170    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da5d1545-9219-479d-9130-98f45e9b6b76-config-out\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.161268 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.161247    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.161715 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.161694    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-config-volume\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.161882 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.161862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da5d1545-9219-479d-9130-98f45e9b6b76-web-config\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.162519 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.162488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.165706 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.165688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw2mb\" (UniqueName: \"kubernetes.io/projected/da5d1545-9219-479d-9130-98f45e9b6b76-kube-api-access-bw2mb\") pod \"alertmanager-main-0\" (UID: \"da5d1545-9219-479d-9130-98f45e9b6b76\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.192128 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.192091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:18:51.476213 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.476054    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:18:51.479884 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:51.479856    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5d1545_9219_479d_9130_98f45e9b6b76.slice/crio-50a389aa7ca9dbe630a6c8399edb32a6eaeb4820eeffd00203d65504f1258744 WatchSource:0}: Error finding container 50a389aa7ca9dbe630a6c8399edb32a6eaeb4820eeffd00203d65504f1258744: Status 404 returned error can't find the container with id 50a389aa7ca9dbe630a6c8399edb32a6eaeb4820eeffd00203d65504f1258744
Sep 11 16:18:51.613801 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.613771    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"50a389aa7ca9dbe630a6c8399edb32a6eaeb4820eeffd00203d65504f1258744"}
Sep 11 16:18:51.615234 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.615200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2cj5" event={"ID":"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1","Type":"ContainerStarted","Data":"0dae82cd70bc1a5c99d88b78822f0eeb43b17a4c0dacfed73688e47a7489cdf3"}
Sep 11 16:18:51.617405 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.617380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2" event={"ID":"5663307d-b526-4545-a021-888486bd380d","Type":"ContainerStarted","Data":"5c83e444cb9054fac108478ee6971c55aff5140fd4f5060901a8f1889d534705"}
Sep 11 16:18:51.617499 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.617423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2" event={"ID":"5663307d-b526-4545-a021-888486bd380d","Type":"ContainerStarted","Data":"ec59af83620a367f018f360c357549a808a471ac7f0490a4082fecfe8f975783"}
Sep 11 16:18:51.617499 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:51.617447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2" event={"ID":"5663307d-b526-4545-a021-888486bd380d","Type":"ContainerStarted","Data":"be24bb156d03c63e8258ec35d7592847ead45c871b70c7c95774004293c37c69"}
Sep 11 16:18:52.626004 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:52.625947    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cd40cd2-32ce-453a-bff5-639d8f9c7ed1" containerID="6aec1f339cb03c7c09ad217a5360a12641309e0a258a709a1ac27f26032128b1" exitCode=0
Sep 11 16:18:52.626431 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:52.626003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2cj5" event={"ID":"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1","Type":"ContainerDied","Data":"6aec1f339cb03c7c09ad217a5360a12641309e0a258a709a1ac27f26032128b1"}
Sep 11 16:18:53.631462 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.631427    2566 generic.go:356] "Generic (PLEG): container finished" podID="da5d1545-9219-479d-9130-98f45e9b6b76" containerID="80550e814276464e37bf78181fd792d884c87e15ce344c5509603b184ea407ae" exitCode=0
Sep 11 16:18:53.631921 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.631517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerDied","Data":"80550e814276464e37bf78181fd792d884c87e15ce344c5509603b184ea407ae"}
Sep 11 16:18:53.634205 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.634181    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2cj5" event={"ID":"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1","Type":"ContainerStarted","Data":"63f4e4af16abe7cebf92d36bbd2657eb2d4ee1d96cbedbdabb43458c3c7fd000"}
Sep 11 16:18:53.634308 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.634230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2cj5" event={"ID":"1cd40cd2-32ce-453a-bff5-639d8f9c7ed1","Type":"ContainerStarted","Data":"1c67cb4278e9b07596e5f88653683ca290368aa1c2f7be7ed0a9fe2ec53286b3"}
Sep 11 16:18:53.636610 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.636591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2" event={"ID":"5663307d-b526-4545-a021-888486bd380d","Type":"ContainerStarted","Data":"95f56b6ffdbd5e57bd063aae8564f54e75c06dab046c14638a50c8e96a613bd7"}
Sep 11 16:18:53.680472 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.680427    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n2cj5" podStartSLOduration=3.572866503 podStartE2EDuration="4.680414599s" totalImagesPullingTime="1.044519161s" totalInitContainerRuntime="63.028935ms" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:53.678808437 +0000 UTC m=+96.094291417" watchObservedRunningTime="2026-09-11 16:18:53.680414599 +0000 UTC m=+96.095897580"
Sep 11 16:18:53.698233 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:53.698197    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sgdb2" podStartSLOduration=3.183321 podStartE2EDuration="4.698187513s" totalImagesPullingTime="1.514866513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:53.696505757 +0000 UTC m=+96.111988740" watchObservedRunningTime="2026-09-11 16:18:53.698187513 +0000 UTC m=+96.113670494"
Sep 11 16:18:54.217352 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.217315    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-794b87fc68-m6shv"]
Sep 11 16:18:54.219345 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.219329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.224303 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224183    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6da7i2nau1ush\""
Sep 11 16:18:54.224303 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224199    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 16:18:54.224303 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224185    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 16:18:54.224524 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224393    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 16:18:54.224524 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224499    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 16:18:54.224599 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.224585    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4cwx8\""
Sep 11 16:18:54.231017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.230991    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-794b87fc68-m6shv"]
Sep 11 16:18:54.282909 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.282877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.282997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-tls\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.283029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-audit-log\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.283061    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-client-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.283124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-client-certs\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.283149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-metrics-server-audit-profiles\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.283237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.283175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs88l\" (UniqueName: \"kubernetes.io/projected/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-kube-api-access-vs88l\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384442 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-tls\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384442 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-audit-log\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-client-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-client-certs\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-metrics-server-audit-profiles\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs88l\" (UniqueName: \"kubernetes.io/projected/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-kube-api-access-vs88l\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.384926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.384901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-audit-log\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.385629 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.385577    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.385887 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.385857    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-metrics-server-audit-profiles\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.387643 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.387618    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-client-certs\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.387739 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.387650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-client-ca-bundle\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.387791 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.387742    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-secret-metrics-server-tls\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.393553 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.393535    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs88l\" (UniqueName: \"kubernetes.io/projected/2bc4a8d0-a075-46b2-8612-5fb6b27723c3-kube-api-access-vs88l\") pod \"metrics-server-794b87fc68-m6shv\" (UID: \"2bc4a8d0-a075-46b2-8612-5fb6b27723c3\") " pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.529364 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.529334    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:18:54.817771 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:54.817737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-794b87fc68-m6shv"]
Sep 11 16:18:54.938978 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:18:54.938935    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc4a8d0_a075_46b2_8612_5fb6b27723c3.slice/crio-8132762e0b4ee84aeb44ac263c8878a1fb06b0bac3499f8f77211c2016a3e266 WatchSource:0}: Error finding container 8132762e0b4ee84aeb44ac263c8878a1fb06b0bac3499f8f77211c2016a3e266: Status 404 returned error can't find the container with id 8132762e0b4ee84aeb44ac263c8878a1fb06b0bac3499f8f77211c2016a3e266
Sep 11 16:18:55.651095 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.650998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"32af300836fea8129c431812b262c77d1a462c0acb509b474199557ec64453bf"}
Sep 11 16:18:55.651257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.651148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"64a43073e5dc98bf03d8b94825c33d914045f2f6580b71a37e764f46081448ce"}
Sep 11 16:18:55.651257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.651194    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"4666242c5936e0fb1babd04feec2655de251e80d8160a0b15fb01e28360d3723"}
Sep 11 16:18:55.651257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.651234    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"da5b58f13617db7c1f46f90a95e3eb6ddc275007557b07b09865b82af026a1e1"}
Sep 11 16:18:55.651397 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.651264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"872693c65c5f10d77e7cf890906cecc3e59ae3cd4960c61e7eb001b61c1ba3fe"}
Sep 11 16:18:55.652467 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:55.652432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv" event={"ID":"2bc4a8d0-a075-46b2-8612-5fb6b27723c3","Type":"ContainerStarted","Data":"8132762e0b4ee84aeb44ac263c8878a1fb06b0bac3499f8f77211c2016a3e266"}
Sep 11 16:18:56.506169 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:56.506139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-87g4d"
Sep 11 16:18:56.658557 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:56.658465    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv" event={"ID":"2bc4a8d0-a075-46b2-8612-5fb6b27723c3","Type":"ContainerStarted","Data":"ae120f8e499b21cb9874c5c6563dcd60611dc19dcc1babacb62a1bcf4cac48af"}
Sep 11 16:18:56.662016 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:56.661991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da5d1545-9219-479d-9130-98f45e9b6b76","Type":"ContainerStarted","Data":"48bbb3721f1acd9aad7ec91af01c7e1a69a13657e2e1010ea6d2094bb690eaca"}
Sep 11 16:18:56.680758 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:56.680714    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv" podStartSLOduration=1.308068643 podStartE2EDuration="2.680699386s" totalImagesPullingTime="1.372630743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:56.678647635 +0000 UTC m=+99.094130615" watchObservedRunningTime="2026-09-11 16:18:56.680699386 +0000 UTC m=+99.096182366"
Sep 11 16:18:56.721252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:18:56.721210    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.09492195 podStartE2EDuration="6.721197187s" totalImagesPullingTime="3.606761216s" totalInitContainerRuntime="19.514021ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:56.720155762 +0000 UTC m=+99.135638744" watchObservedRunningTime="2026-09-11 16:18:56.721197187 +0000 UTC m=+99.136680234"
Sep 11 16:19:06.702937 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:06.702859    2566 generic.go:356] "Generic (PLEG): container finished" podID="6eac51b9-5627-4d01-b33c-efb8655ab211" containerID="54b1f32feceb7b2cddcaf56c90d456bf5f767f9ee3ce2526d126374700705e27" exitCode=0
Sep 11 16:19:06.702937 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:06.702914    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c" event={"ID":"6eac51b9-5627-4d01-b33c-efb8655ab211","Type":"ContainerDied","Data":"54b1f32feceb7b2cddcaf56c90d456bf5f767f9ee3ce2526d126374700705e27"}
Sep 11 16:19:06.703370 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:06.703221    2566 scope.go:117] "RemoveContainer" containerID="54b1f32feceb7b2cddcaf56c90d456bf5f767f9ee3ce2526d126374700705e27"
Sep 11 16:19:07.708810 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:07.708775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jsz5c" event={"ID":"6eac51b9-5627-4d01-b33c-efb8655ab211","Type":"ContainerStarted","Data":"f62ea9b256ed8d72cca98150765859fa5c22986fff33c984e4195c25669dacc0"}
Sep 11 16:19:11.726033 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:11.725995    2566 generic.go:356] "Generic (PLEG): container finished" podID="b1766afd-bc87-4584-9d5d-4d2a8b77711c" containerID="814dc9e3a59741fef90074b91eb96641e7ae2427920663e4745e79c68095acae" exitCode=0
Sep 11 16:19:11.726397 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:11.726070    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-lzxh9" event={"ID":"b1766afd-bc87-4584-9d5d-4d2a8b77711c","Type":"ContainerDied","Data":"814dc9e3a59741fef90074b91eb96641e7ae2427920663e4745e79c68095acae"}
Sep 11 16:19:11.726454 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:11.726440    2566 scope.go:117] "RemoveContainer" containerID="814dc9e3a59741fef90074b91eb96641e7ae2427920663e4745e79c68095acae"
Sep 11 16:19:12.731390 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:12.731357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-lzxh9" event={"ID":"b1766afd-bc87-4584-9d5d-4d2a8b77711c","Type":"ContainerStarted","Data":"40ac02499e242f3a3998b9d1a668c5a4a859076846df44a298796e0f609e1705"}
Sep 11 16:19:13.228028 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.227980    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86767db879-d6gml" podUID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" containerName="registry" containerID="cri-o://10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f" gracePeriod=30
Sep 11 16:19:13.470861 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.470841    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:19:13.544455 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544580 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544465    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544580 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544499    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544580 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544541    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544580 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544558    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544834 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544584    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544834 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544720    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spktn\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.544834 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.544784    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted\") pod \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\" (UID: \"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0\") "
Sep 11 16:19:13.545480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.545225    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:13.545480 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.545362    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:13.547552 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.547507    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:19:13.547698 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.547549    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:19:13.547817 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.547788    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn" (OuterVolumeSpecName: "kube-api-access-spktn") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "kube-api-access-spktn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:19:13.548145 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.548121    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:19:13.548248 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.548226    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:19:13.556043 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.556015    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" (UID: "1bcb1516-9c13-4691-9f3f-aa1e13b51ed0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:19:13.646221 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646197    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-bound-sa-token\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646221 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646220    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-certificates\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646232    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-image-registry-private-configuration\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646241    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-registry-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646249    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-trusted-ca\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646257    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-installation-pull-secrets\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646264    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spktn\" (UniqueName: \"kubernetes.io/projected/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-kube-api-access-spktn\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.646341 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.646272    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0-ca-trust-extracted\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:19:13.737360 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.737329    2566 generic.go:356] "Generic (PLEG): container finished" podID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" containerID="10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f" exitCode=0
Sep 11 16:19:13.737779 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.737373    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86767db879-d6gml"
Sep 11 16:19:13.737779 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.737411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86767db879-d6gml" event={"ID":"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0","Type":"ContainerDied","Data":"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"}
Sep 11 16:19:13.737779 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.737477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86767db879-d6gml" event={"ID":"1bcb1516-9c13-4691-9f3f-aa1e13b51ed0","Type":"ContainerDied","Data":"31424be40847d58753b2f7ca5f5bccc70415ef3ebcf4ab63406df4f98764a7d7"}
Sep 11 16:19:13.737779 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.737511    2566 scope.go:117] "RemoveContainer" containerID="10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"
Sep 11 16:19:13.746158 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.746142    2566 scope.go:117] "RemoveContainer" containerID="10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"
Sep 11 16:19:13.746418 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:19:13.746400    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f\": container with ID starting with 10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f not found: ID does not exist" containerID="10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"
Sep 11 16:19:13.746481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.746425    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f"} err="failed to get container status \"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f\": rpc error: code = NotFound desc = could not find container \"10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f\": container with ID starting with 10ecda7f716249be811aac82171fe82b0a4e9b1ed2cea92cbc925856c70a881f not found: ID does not exist"
Sep 11 16:19:13.760271 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.760251    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:19:13.764481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:13.764460    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86767db879-d6gml"]
Sep 11 16:19:14.110926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:14.110897    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" path="/var/lib/kubelet/pods/1bcb1516-9c13-4691-9f3f-aa1e13b51ed0/volumes"
Sep 11 16:19:14.529596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:14.529571    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:19:14.529725 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:14.529638    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:19:16.751989 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:16.751940    2566 generic.go:356] "Generic (PLEG): container finished" podID="e58035b6-c5c3-434c-8a77-060704f45d0b" containerID="6a2bcfc4d7eca4c82a44d68da874cd8f14b058f9d34cb6815dc1cd3bdef68b5c" exitCode=0
Sep 11 16:19:16.752337 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:16.752004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj" event={"ID":"e58035b6-c5c3-434c-8a77-060704f45d0b","Type":"ContainerDied","Data":"6a2bcfc4d7eca4c82a44d68da874cd8f14b058f9d34cb6815dc1cd3bdef68b5c"}
Sep 11 16:19:16.752337 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:16.752306    2566 scope.go:117] "RemoveContainer" containerID="6a2bcfc4d7eca4c82a44d68da874cd8f14b058f9d34cb6815dc1cd3bdef68b5c"
Sep 11 16:19:17.759031 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:17.758993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cdlgj" event={"ID":"e58035b6-c5c3-434c-8a77-060704f45d0b","Type":"ContainerStarted","Data":"1c944d0ca9b45dfd0d161b6f62f32465401ba5d2b36ba27d3abb4c3860c22a9f"}
Sep 11 16:19:34.535395 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:34.535368    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:19:34.539270 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:19:34.539248    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-794b87fc68-m6shv"
Sep 11 16:20:14.184696 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.184622    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"]
Sep 11 16:20:14.185105 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.184916    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" containerName="registry"
Sep 11 16:20:14.185105 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.184928    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" containerName="registry"
Sep 11 16:20:14.185105 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.185032    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bcb1516-9c13-4691-9f3f-aa1e13b51ed0" containerName="registry"
Sep 11 16:20:14.188265 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.188245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.191994 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.191940    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 16:20:14.192176 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.191950    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-45j7l\""
Sep 11 16:20:14.192374 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.192353    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 16:20:14.192495 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.192208    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 16:20:14.192495 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.192363    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 16:20:14.192495 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.192128    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 16:20:14.198641 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.198618    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 16:20:14.202776 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.202754    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"]
Sep 11 16:20:14.218250 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-serving-certs-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218375 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-metrics-client-ca\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218366    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-federate-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfdg\" (UniqueName: \"kubernetes.io/projected/ad7f59b0-b129-439c-a3bd-765b7a3023d8-kube-api-access-kvfdg\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.218606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.218591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.319892 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.319860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-metrics-client-ca\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.319892 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.319898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-federate-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320131 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.319919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320131 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.319944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320131 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.319990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320131 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.320018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfdg\" (UniqueName: \"kubernetes.io/projected/ad7f59b0-b129-439c-a3bd-765b7a3023d8-kube-api-access-kvfdg\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320313 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.320163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320559 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.320531    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-serving-certs-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.320781 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.320753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-metrics-client-ca\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.321140 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.321113    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.321241 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.321209    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f59b0-b129-439c-a3bd-765b7a3023d8-serving-certs-ca-bundle\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.322409 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.322388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-federate-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.322523 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.322500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.322733 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.322715    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-telemeter-client-tls\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.322847 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.322825    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad7f59b0-b129-439c-a3bd-765b7a3023d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.329602 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.329582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfdg\" (UniqueName: \"kubernetes.io/projected/ad7f59b0-b129-439c-a3bd-765b7a3023d8-kube-api-access-kvfdg\") pod \"telemeter-client-7f57dd5d9c-bnvbp\" (UID: \"ad7f59b0-b129-439c-a3bd-765b7a3023d8\") " pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.498295 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.498209    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"
Sep 11 16:20:14.782006 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.781978    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp"]
Sep 11 16:20:14.782913 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:20:14.782888    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7f59b0_b129_439c_a3bd_765b7a3023d8.slice/crio-c8de5d39d0de8985ef373892e8cf6c0ada60cab3f673f4b309fb6426d497bf7f WatchSource:0}: Error finding container c8de5d39d0de8985ef373892e8cf6c0ada60cab3f673f4b309fb6426d497bf7f: Status 404 returned error can't find the container with id c8de5d39d0de8985ef373892e8cf6c0ada60cab3f673f4b309fb6426d497bf7f
Sep 11 16:20:14.990228 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:14.990193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp" event={"ID":"ad7f59b0-b129-439c-a3bd-765b7a3023d8","Type":"ContainerStarted","Data":"c8de5d39d0de8985ef373892e8cf6c0ada60cab3f673f4b309fb6426d497bf7f"}
Sep 11 16:20:16.998920 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:16.998895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp" event={"ID":"ad7f59b0-b129-439c-a3bd-765b7a3023d8","Type":"ContainerStarted","Data":"e96a9df40f47a7034b5351dc0f3919ce05c51d89d0e0f73c34354a4e6858b750"}
Sep 11 16:20:18.004551 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:18.004517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp" event={"ID":"ad7f59b0-b129-439c-a3bd-765b7a3023d8","Type":"ContainerStarted","Data":"0162e2ad036580bd04dfdac7094070c4fae72317f8abf1347a5a75075ec9951f"}
Sep 11 16:20:18.005030 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:18.004573    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp" event={"ID":"ad7f59b0-b129-439c-a3bd-765b7a3023d8","Type":"ContainerStarted","Data":"25d8b99f52c4c280f7223a27e6de115896e62dbce679f2503db531093193a4bb"}
Sep 11 16:20:18.031916 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:20:18.031871    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f57dd5d9c-bnvbp" podStartSLOduration=1.935203219 podStartE2EDuration="4.031854385s" totalImagesPullingTime="2.096651166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:20:18.030227024 +0000 UTC m=+180.445710032" watchObservedRunningTime="2026-09-11 16:20:18.031854385 +0000 UTC m=+180.447337365"
Sep 11 16:22:11.984985 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.984939    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"]
Sep 11 16:22:11.988402 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.988385    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:11.991398 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.991378    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 16:22:11.991519 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.991447    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hn8k7\""
Sep 11 16:22:11.992425 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.992409    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 16:22:11.998137 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:11.998113    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"]
Sep 11 16:22:12.059090 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.059056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.059090 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.059097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6gj\" (UniqueName: \"kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.059287 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.059206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.159739 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.159707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.159896 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.159782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.159896 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.159812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6gj\" (UniqueName: \"kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.160230 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.160206    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.160316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.160240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.168726 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.168704    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6gj\" (UniqueName: \"kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.297735 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.297709    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:12.567476 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:12.567450    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"]
Sep 11 16:22:12.570202 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:22:12.570171    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbf8ad3_7470_4868_b75f_f40d2dc09345.slice/crio-ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9 WatchSource:0}: Error finding container ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9: Status 404 returned error can't find the container with id ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9
Sep 11 16:22:13.469981 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:13.469927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l" event={"ID":"0dbf8ad3-7470-4868-b75f-f40d2dc09345","Type":"ContainerStarted","Data":"ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9"}
Sep 11 16:22:18.595759 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:18.595734    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:22:18.597990 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:18.595786    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:22:18.605272 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:18.605251    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 16:22:19.500300 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:19.500262    2566 generic.go:356] "Generic (PLEG): container finished" podID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerID="b53bfeb8d8b7afde5a84f983cb52f0934ca25134c7bd8fcf938879790b2522cc" exitCode=0
Sep 11 16:22:19.500485 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:19.500342    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l" event={"ID":"0dbf8ad3-7470-4868-b75f-f40d2dc09345","Type":"ContainerDied","Data":"b53bfeb8d8b7afde5a84f983cb52f0934ca25134c7bd8fcf938879790b2522cc"}
Sep 11 16:22:19.501340 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:19.501324    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:22:21.509820 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:21.509783    2566 generic.go:356] "Generic (PLEG): container finished" podID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerID="296b95b23b8028775b60329ef76e2ee2bf4e636063ec6695992303f9b3e7b3cf" exitCode=0
Sep 11 16:22:21.510179 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:21.509877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l" event={"ID":"0dbf8ad3-7470-4868-b75f-f40d2dc09345","Type":"ContainerDied","Data":"296b95b23b8028775b60329ef76e2ee2bf4e636063ec6695992303f9b3e7b3cf"}
Sep 11 16:22:29.549387 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:29.549351    2566 generic.go:356] "Generic (PLEG): container finished" podID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerID="2a4938a1ec4de26a2a0bfb6c627b53f0f19b5fefc3dd6a97b81bfcfbbf0d226a" exitCode=0
Sep 11 16:22:29.549810 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:29.549421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l" event={"ID":"0dbf8ad3-7470-4868-b75f-f40d2dc09345","Type":"ContainerDied","Data":"2a4938a1ec4de26a2a0bfb6c627b53f0f19b5fefc3dd6a97b81bfcfbbf0d226a"}
Sep 11 16:22:30.692427 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.692400    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:30.823752 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.823658    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w6gj\" (UniqueName: \"kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj\") pod \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") "
Sep 11 16:22:30.823752 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.823732    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util\") pod \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") "
Sep 11 16:22:30.823937 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.823777    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle\") pod \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\" (UID: \"0dbf8ad3-7470-4868-b75f-f40d2dc09345\") "
Sep 11 16:22:30.824504 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.824470    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle" (OuterVolumeSpecName: "bundle") pod "0dbf8ad3-7470-4868-b75f-f40d2dc09345" (UID: "0dbf8ad3-7470-4868-b75f-f40d2dc09345"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:22:30.825939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.825913    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj" (OuterVolumeSpecName: "kube-api-access-6w6gj") pod "0dbf8ad3-7470-4868-b75f-f40d2dc09345" (UID: "0dbf8ad3-7470-4868-b75f-f40d2dc09345"). InnerVolumeSpecName "kube-api-access-6w6gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:22:30.828587 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.828562    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util" (OuterVolumeSpecName: "util") pod "0dbf8ad3-7470-4868-b75f-f40d2dc09345" (UID: "0dbf8ad3-7470-4868-b75f-f40d2dc09345"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:22:30.925384 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.925344    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w6gj\" (UniqueName: \"kubernetes.io/projected/0dbf8ad3-7470-4868-b75f-f40d2dc09345-kube-api-access-6w6gj\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:22:30.925384 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.925377    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-util\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:22:30.925384 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:30.925387    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf8ad3-7470-4868-b75f-f40d2dc09345-bundle\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:22:31.566584 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:31.566543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l" event={"ID":"0dbf8ad3-7470-4868-b75f-f40d2dc09345","Type":"ContainerDied","Data":"ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9"}
Sep 11 16:22:31.566584 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:31.566585    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ac9n7l"
Sep 11 16:22:31.566800 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:31.566601    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec49dc0f804eb293277bb650a5bc0e4572b8d2cf7f7b07b932335f0575dba3e9"
Sep 11 16:22:35.003147 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003110    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"]
Sep 11 16:22:35.003525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003469    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="pull"
Sep 11 16:22:35.003525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003482    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="pull"
Sep 11 16:22:35.003525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003490    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="extract"
Sep 11 16:22:35.003525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003498    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="extract"
Sep 11 16:22:35.003525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003523    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="util"
Sep 11 16:22:35.003734 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003529    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="util"
Sep 11 16:22:35.003734 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.003585    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf8ad3-7470-4868-b75f-f40d2dc09345" containerName="extract"
Sep 11 16:22:35.041840 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.041805    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"]
Sep 11 16:22:35.042027 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.041918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.045024 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.044998    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 11 16:22:35.045172 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.045029    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-w9lhz\""
Sep 11 16:22:35.045172 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.045033    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 11 16:22:35.045172 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.045003    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 11 16:22:35.164212 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.164171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-222w7\" (UniqueName: \"kubernetes.io/projected/5c0cc64e-1395-4968-ab1e-e9e84dde786a-kube-api-access-222w7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.164212 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.164216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/5c0cc64e-1395-4968-ab1e-e9e84dde786a-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.264855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.264766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-222w7\" (UniqueName: \"kubernetes.io/projected/5c0cc64e-1395-4968-ab1e-e9e84dde786a-kube-api-access-222w7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.264855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.264820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/5c0cc64e-1395-4968-ab1e-e9e84dde786a-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.267298 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.267272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/5c0cc64e-1395-4968-ab1e-e9e84dde786a-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.273937 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.273911    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-222w7\" (UniqueName: \"kubernetes.io/projected/5c0cc64e-1395-4968-ab1e-e9e84dde786a-kube-api-access-222w7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-9gqvt\" (UID: \"5c0cc64e-1395-4968-ab1e-e9e84dde786a\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.352495 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.352455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:35.646802 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:35.646769    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"]
Sep 11 16:22:35.648442 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:22:35.648408    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0cc64e_1395_4968_ab1e_e9e84dde786a.slice/crio-913d2e3d65980ee82e8319d8f7f93a313fbe36db4aad4874ae1e19407f41cb71 WatchSource:0}: Error finding container 913d2e3d65980ee82e8319d8f7f93a313fbe36db4aad4874ae1e19407f41cb71: Status 404 returned error can't find the container with id 913d2e3d65980ee82e8319d8f7f93a313fbe36db4aad4874ae1e19407f41cb71
Sep 11 16:22:36.590179 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:36.590137    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt" event={"ID":"5c0cc64e-1395-4968-ab1e-e9e84dde786a","Type":"ContainerStarted","Data":"913d2e3d65980ee82e8319d8f7f93a313fbe36db4aad4874ae1e19407f41cb71"}
Sep 11 16:22:39.587898 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.587860    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-6bkjj"]
Sep 11 16:22:39.604611 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.604583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-6bkjj"]
Sep 11 16:22:39.604771 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.604684    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.607441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.607398    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt" event={"ID":"5c0cc64e-1395-4968-ab1e-e9e84dde786a","Type":"ContainerStarted","Data":"985095ad3f1eab94a7a913d590487a8423cff7072a779d85a2a917be100e0489"}
Sep 11 16:22:39.607603 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.607585    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:22:39.609518 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.609489    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-hh4mp\""
Sep 11 16:22:39.609983 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.609950    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 11 16:22:39.610677 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.610663    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 11 16:22:39.691786 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.691688    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt" podStartSLOduration=2.543698816 podStartE2EDuration="5.691666494s" totalImagesPullingTime="3.147967678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:22:39.691445449 +0000 UTC m=+322.106928432" watchObservedRunningTime="2026-09-11 16:22:39.691666494 +0000 UTC m=+322.107149497"
Sep 11 16:22:39.704371 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.704332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.704566 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.704473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9zh9\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-kube-api-access-x9zh9\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.704566 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.704518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6ba1c52c-c973-44df-b324-af0a029722e2-cabundle0\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.805199 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.805163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9zh9\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-kube-api-access-x9zh9\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.805199 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.805211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6ba1c52c-c973-44df-b324-af0a029722e2-cabundle0\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.805455 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.805279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.805455 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:22:39.805402    2566 secret.go:281] references non-existent secret key: ca.crt
Sep 11 16:22:39.805455 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:22:39.805417    2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 11 16:22:39.805455 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:22:39.805425    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-6bkjj: references non-existent secret key: ca.crt
Sep 11 16:22:39.805636 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:22:39.805478    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates podName:6ba1c52c-c973-44df-b324-af0a029722e2 nodeName:}" failed. No retries permitted until 2026-09-11 16:22:40.305459962 +0000 UTC m=+322.720942922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates") pod "keda-operator-579c8958f7-6bkjj" (UID: "6ba1c52c-c973-44df-b324-af0a029722e2") : references non-existent secret key: ca.crt
Sep 11 16:22:39.805861 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.805843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6ba1c52c-c973-44df-b324-af0a029722e2-cabundle0\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:39.814593 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:39.814566    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9zh9\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-kube-api-access-x9zh9\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:40.245559 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.245521    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"]
Sep 11 16:22:40.248332 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.248308    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.253154 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.253123    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 11 16:22:40.269429 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.269400    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"]
Sep 11 16:22:40.310583 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.310543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:40.312941 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.312919    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6ba1c52c-c973-44df-b324-af0a029722e2-certificates\") pod \"keda-operator-579c8958f7-6bkjj\" (UID: \"6ba1c52c-c973-44df-b324-af0a029722e2\") " pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:40.411819 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.411778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-certificates\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.411819 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.411823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmppx\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-kube-api-access-vmppx\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.412066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.411890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f7fb18aa-2cff-41c7-8066-75d33beb5c33-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.512549 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.512439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-certificates\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.512549 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.512490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmppx\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-kube-api-access-vmppx\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.512783 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.512554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f7fb18aa-2cff-41c7-8066-75d33beb5c33-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.513079 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.513058    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f7fb18aa-2cff-41c7-8066-75d33beb5c33-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.515158 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.515138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-certificates\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.515828 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.515809    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:40.537667 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.537637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmppx\" (UniqueName: \"kubernetes.io/projected/f7fb18aa-2cff-41c7-8066-75d33beb5c33-kube-api-access-vmppx\") pod \"keda-metrics-apiserver-77857b7b98-fbgrf\" (UID: \"f7fb18aa-2cff-41c7-8066-75d33beb5c33\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.560321 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.560278    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:40.794057 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.794030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-6bkjj"]
Sep 11 16:22:40.795717 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:22:40.795692    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba1c52c_c973_44df_b324_af0a029722e2.slice/crio-19d1ca807a1ff3dd3f70d0b88b149678d1591e176b80a6f982d08403a0ed974d WatchSource:0}: Error finding container 19d1ca807a1ff3dd3f70d0b88b149678d1591e176b80a6f982d08403a0ed974d: Status 404 returned error can't find the container with id 19d1ca807a1ff3dd3f70d0b88b149678d1591e176b80a6f982d08403a0ed974d
Sep 11 16:22:40.850673 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:40.850621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"]
Sep 11 16:22:40.853656 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:22:40.853627    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fb18aa_2cff_41c7_8066_75d33beb5c33.slice/crio-193c158ac495da3d37d029dcccfeae0073954119ad7ebb92a3a7e79e067891d2 WatchSource:0}: Error finding container 193c158ac495da3d37d029dcccfeae0073954119ad7ebb92a3a7e79e067891d2: Status 404 returned error can't find the container with id 193c158ac495da3d37d029dcccfeae0073954119ad7ebb92a3a7e79e067891d2
Sep 11 16:22:41.620633 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:41.620548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-6bkjj" event={"ID":"6ba1c52c-c973-44df-b324-af0a029722e2","Type":"ContainerStarted","Data":"19d1ca807a1ff3dd3f70d0b88b149678d1591e176b80a6f982d08403a0ed974d"}
Sep 11 16:22:41.622278 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:41.622243    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf" event={"ID":"f7fb18aa-2cff-41c7-8066-75d33beb5c33","Type":"ContainerStarted","Data":"193c158ac495da3d37d029dcccfeae0073954119ad7ebb92a3a7e79e067891d2"}
Sep 11 16:22:44.639863 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.639757    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-6bkjj" event={"ID":"6ba1c52c-c973-44df-b324-af0a029722e2","Type":"ContainerStarted","Data":"93be733136436d9592400232b93c55db8fac2414491d621c23c3f4c550965131"}
Sep 11 16:22:44.640326 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.639946    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:22:44.641564 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.641543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf" event={"ID":"f7fb18aa-2cff-41c7-8066-75d33beb5c33","Type":"ContainerStarted","Data":"90ce89ede1ab77f83fd3d1b4043cfd2df109472ee46414274d181c3b1452b480"}
Sep 11 16:22:44.641703 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.641683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:22:44.702313 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.702261    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-6bkjj" podStartSLOduration=2.153805338 podStartE2EDuration="5.702243005s" totalImagesPullingTime="3.548437667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:22:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:22:44.676064858 +0000 UTC m=+327.091547839" watchObservedRunningTime="2026-09-11 16:22:44.702243005 +0000 UTC m=+327.117725985"
Sep 11 16:22:44.702529 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:44.702504    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf" podStartSLOduration=1.215918166 podStartE2EDuration="4.702495516s" totalImagesPullingTime="3.48657735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:22:44.700216284 +0000 UTC m=+327.115699267" watchObservedRunningTime="2026-09-11 16:22:44.702495516 +0000 UTC m=+327.117978500"
Sep 11 16:22:55.651444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:22:55.651411    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-fbgrf"
Sep 11 16:23:00.615466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:00.615432    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-9gqvt"
Sep 11 16:23:05.649659 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:05.649628    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-6bkjj"
Sep 11 16:23:58.266989 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.266940    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:23:58.269212 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.269192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.272656 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.272638    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-zpsxt\""
Sep 11 16:23:58.272766 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.272663    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 11 16:23:58.273599 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.273583    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 11 16:23:58.273667 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.273582    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 11 16:23:58.284454 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.284433    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"]
Sep 11 16:23:58.286658 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.286640    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:23:58.286730 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.286716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.289928 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.289910    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 11 16:23:58.290044 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.289928    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-nnwsf\""
Sep 11 16:23:58.302392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.302369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"]
Sep 11 16:23:58.385022 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.384999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvcr5\" (UniqueName: \"kubernetes.io/projected/54db910f-d070-4637-84b9-e9ae0d8bbe20-kube-api-access-kvcr5\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.385153 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.385046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5wdq\" (UniqueName: \"kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.385153 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.385078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.385153 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.385134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54db910f-d070-4637-84b9-e9ae0d8bbe20-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.486294 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.486267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvcr5\" (UniqueName: \"kubernetes.io/projected/54db910f-d070-4637-84b9-e9ae0d8bbe20-kube-api-access-kvcr5\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.486403 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.486312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5wdq\" (UniqueName: \"kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.486403 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.486334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.486403 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.486372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54db910f-d070-4637-84b9-e9ae0d8bbe20-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.488823 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.488802    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54db910f-d070-4637-84b9-e9ae0d8bbe20-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.488910 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.488875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.494522 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.494497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5wdq\" (UniqueName: \"kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq\") pod \"kserve-controller-manager-656b75fc8d-p7pw9\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") " pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.494799 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.494779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvcr5\" (UniqueName: \"kubernetes.io/projected/54db910f-d070-4637-84b9-e9ae0d8bbe20-kube-api-access-kvcr5\") pod \"llmisvc-controller-manager-645f9c8fd6-6xcgg\" (UID: \"54db910f-d070-4637-84b9-e9ae0d8bbe20\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.579397 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.579378    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:23:58.596334 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.596308    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:23:58.849441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.849419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:23:58.854283 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:23:58.852192    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3423135_790c_4bcb_900c_602d3970973c.slice/crio-952414ded6587712767a83d4ce87a273ff5d1f46dc7b351f0cc1963981c5fd64 WatchSource:0}: Error finding container 952414ded6587712767a83d4ce87a273ff5d1f46dc7b351f0cc1963981c5fd64: Status 404 returned error can't find the container with id 952414ded6587712767a83d4ce87a273ff5d1f46dc7b351f0cc1963981c5fd64
Sep 11 16:23:58.876179 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.876159    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"]
Sep 11 16:23:58.878076 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:23:58.878050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54db910f_d070_4637_84b9_e9ae0d8bbe20.slice/crio-d8013d1afe41951a467965055c911edccc772f9451ff57b7003f6b758caeff47 WatchSource:0}: Error finding container d8013d1afe41951a467965055c911edccc772f9451ff57b7003f6b758caeff47: Status 404 returned error can't find the container with id d8013d1afe41951a467965055c911edccc772f9451ff57b7003f6b758caeff47
Sep 11 16:23:58.988192 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.988158    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" event={"ID":"d3423135-790c-4bcb-900c-602d3970973c","Type":"ContainerStarted","Data":"952414ded6587712767a83d4ce87a273ff5d1f46dc7b351f0cc1963981c5fd64"}
Sep 11 16:23:58.989395 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:23:58.989369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg" event={"ID":"54db910f-d070-4637-84b9-e9ae0d8bbe20","Type":"ContainerStarted","Data":"d8013d1afe41951a467965055c911edccc772f9451ff57b7003f6b758caeff47"}
Sep 11 16:24:03.013897 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.013853    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" event={"ID":"d3423135-790c-4bcb-900c-602d3970973c","Type":"ContainerStarted","Data":"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"}
Sep 11 16:24:03.014356 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.013945    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:24:03.015457 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.015428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg" event={"ID":"54db910f-d070-4637-84b9-e9ae0d8bbe20","Type":"ContainerStarted","Data":"541cc8179b9c487aa32c4b572e7a37546d80ab25a4ee3789a8887e4de1aff4b4"}
Sep 11 16:24:03.015568 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.015542    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:24:03.037074 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.037031    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" podStartSLOduration=1.8542560639999999 podStartE2EDuration="5.037019072s" totalImagesPullingTime="3.182763008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:24:03.034599225 +0000 UTC m=+405.450082209" watchObservedRunningTime="2026-09-11 16:24:03.037019072 +0000 UTC m=+405.452502053"
Sep 11 16:24:03.055293 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:03.055251    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg" podStartSLOduration=1.89943876 podStartE2EDuration="5.055238729s" totalImagesPullingTime="3.155799969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:24:03.053141053 +0000 UTC m=+405.468624038" watchObservedRunningTime="2026-09-11 16:24:03.055238729 +0000 UTC m=+405.470721709"
Sep 11 16:24:34.022589 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:34.022557    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-6xcgg"
Sep 11 16:24:34.025630 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:34.025610    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:24:42.168162 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.168071    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:24:42.168549 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.168278    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" podUID="d3423135-790c-4bcb-900c-602d3970973c" containerName="manager" containerID="cri-o://3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd" gracePeriod=10
Sep 11 16:24:42.190510 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.190481    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-svtds"]
Sep 11 16:24:42.192646 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.192628    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.202680 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.202653    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-svtds"]
Sep 11 16:24:42.230259 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.230226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-286pw\" (UniqueName: \"kubernetes.io/projected/d8008d68-df77-47fb-98e7-405ccfd32ec8-kube-api-access-286pw\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.230259 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.230261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8008d68-df77-47fb-98e7-405ccfd32ec8-cert\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.330852 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.330816    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-286pw\" (UniqueName: \"kubernetes.io/projected/d8008d68-df77-47fb-98e7-405ccfd32ec8-kube-api-access-286pw\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.330852 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.330855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8008d68-df77-47fb-98e7-405ccfd32ec8-cert\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.333401 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.333373    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8008d68-df77-47fb-98e7-405ccfd32ec8-cert\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.339844 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.339816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-286pw\" (UniqueName: \"kubernetes.io/projected/d8008d68-df77-47fb-98e7-405ccfd32ec8-kube-api-access-286pw\") pod \"kserve-controller-manager-656b75fc8d-svtds\" (UID: \"d8008d68-df77-47fb-98e7-405ccfd32ec8\") " pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.407618 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.407590    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:24:42.431689 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.431620    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert\") pod \"d3423135-790c-4bcb-900c-602d3970973c\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") "
Sep 11 16:24:42.431689 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.431670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5wdq\" (UniqueName: \"kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq\") pod \"d3423135-790c-4bcb-900c-602d3970973c\" (UID: \"d3423135-790c-4bcb-900c-602d3970973c\") "
Sep 11 16:24:42.434331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.434296    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq" (OuterVolumeSpecName: "kube-api-access-l5wdq") pod "d3423135-790c-4bcb-900c-602d3970973c" (UID: "d3423135-790c-4bcb-900c-602d3970973c"). InnerVolumeSpecName "kube-api-access-l5wdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:24:42.434500 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.434477    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert" (OuterVolumeSpecName: "cert") pod "d3423135-790c-4bcb-900c-602d3970973c" (UID: "d3423135-790c-4bcb-900c-602d3970973c"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:24:42.533311 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.533256    2566 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3423135-790c-4bcb-900c-602d3970973c-cert\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:24:42.533311 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.533304    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5wdq\" (UniqueName: \"kubernetes.io/projected/d3423135-790c-4bcb-900c-602d3970973c-kube-api-access-l5wdq\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:24:42.557199 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.557166    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:42.826555 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:42.826529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-svtds"]
Sep 11 16:24:42.828712 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:24:42.828685    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8008d68_df77_47fb_98e7_405ccfd32ec8.slice/crio-6267cedac9f78c02e6367ce2107f001e57245237468b447c246a47ef4930df5e WatchSource:0}: Error finding container 6267cedac9f78c02e6367ce2107f001e57245237468b447c246a47ef4930df5e: Status 404 returned error can't find the container with id 6267cedac9f78c02e6367ce2107f001e57245237468b447c246a47ef4930df5e
Sep 11 16:24:43.204203 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.204154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-svtds" event={"ID":"d8008d68-df77-47fb-98e7-405ccfd32ec8","Type":"ContainerStarted","Data":"6267cedac9f78c02e6367ce2107f001e57245237468b447c246a47ef4930df5e"}
Sep 11 16:24:43.205481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.205456    2566 generic.go:356] "Generic (PLEG): container finished" podID="d3423135-790c-4bcb-900c-602d3970973c" containerID="3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd" exitCode=0
Sep 11 16:24:43.205596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.205500    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9"
Sep 11 16:24:43.205596 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.205538    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" event={"ID":"d3423135-790c-4bcb-900c-602d3970973c","Type":"ContainerDied","Data":"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"}
Sep 11 16:24:43.205680 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.205592    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-p7pw9" event={"ID":"d3423135-790c-4bcb-900c-602d3970973c","Type":"ContainerDied","Data":"952414ded6587712767a83d4ce87a273ff5d1f46dc7b351f0cc1963981c5fd64"}
Sep 11 16:24:43.205680 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.205627    2566 scope.go:117] "RemoveContainer" containerID="3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"
Sep 11 16:24:43.215323 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.215306    2566 scope.go:117] "RemoveContainer" containerID="3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"
Sep 11 16:24:43.215645 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:24:43.215622    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd\": container with ID starting with 3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd not found: ID does not exist" containerID="3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"
Sep 11 16:24:43.215743 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.215650    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd"} err="failed to get container status \"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd\": rpc error: code = NotFound desc = could not find container \"3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd\": container with ID starting with 3b81cc722f0d758102c2af372d71d5732d3c1e7b8d6dbd4e13e106aa434ae5bd not found: ID does not exist"
Sep 11 16:24:43.226903 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.226876    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:24:43.231540 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:43.231521    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-656b75fc8d-p7pw9"]
Sep 11 16:24:44.112112 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:44.112069    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3423135-790c-4bcb-900c-602d3970973c" path="/var/lib/kubelet/pods/d3423135-790c-4bcb-900c-602d3970973c/volumes"
Sep 11 16:24:44.212739 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:44.212700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-656b75fc8d-svtds" event={"ID":"d8008d68-df77-47fb-98e7-405ccfd32ec8","Type":"ContainerStarted","Data":"5ba112bcf2f9270c39fc9359f4ca19a88867aaca07265c9f5553755def510953"}
Sep 11 16:24:44.213216 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:44.212854    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:24:44.232135 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:24:44.232080    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-656b75fc8d-svtds" podStartSLOduration=1.776049216 podStartE2EDuration="2.232064734s" totalImagesPullingTime="456.015518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:24:44.230006351 +0000 UTC m=+446.645489328" watchObservedRunningTime="2026-09-11 16:24:44.232064734 +0000 UTC m=+446.647547715"
Sep 11 16:25:15.223370 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:15.223340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-656b75fc8d-svtds"
Sep 11 16:25:16.539125 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.539087    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-mrntx"]
Sep 11 16:25:16.539492 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.539478    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3423135-790c-4bcb-900c-602d3970973c" containerName="manager"
Sep 11 16:25:16.539536 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.539494    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3423135-790c-4bcb-900c-602d3970973c" containerName="manager"
Sep 11 16:25:16.539570 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.539557    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3423135-790c-4bcb-900c-602d3970973c" containerName="manager"
Sep 11 16:25:16.542686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.542659    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.545713 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.545688    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-pd28j\""
Sep 11 16:25:16.545829 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.545736    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 11 16:25:16.555010 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.554990    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-mrntx"]
Sep 11 16:25:16.558217 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.558195    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-lkjvs"]
Sep 11 16:25:16.561928 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.561908    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.565551 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.565529    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 11 16:25:16.565679 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.565530    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 11 16:25:16.565859 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.565836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-79h5x\""
Sep 11 16:25:16.573274 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.573253    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-lkjvs"]
Sep 11 16:25:16.627471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.627444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-tls-certs\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.627637 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.627555    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xs7x\" (UniqueName: \"kubernetes.io/projected/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-kube-api-access-9xs7x\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.728563 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.728523    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwlsr\" (UniqueName: \"kubernetes.io/projected/f1a54806-fb78-4ac6-8925-9a4fd45c2443-kube-api-access-qwlsr\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.728759 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.728601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xs7x\" (UniqueName: \"kubernetes.io/projected/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-kube-api-access-9xs7x\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.728759 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.728628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-cert\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.728759 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.728657    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-metrics-certs\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.728759 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.728718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-tls-certs\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.731424 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.731397    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-tls-certs\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.737806 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.737768    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xs7x\" (UniqueName: \"kubernetes.io/projected/b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd-kube-api-access-9xs7x\") pod \"model-serving-api-7b954ffc95-mrntx\" (UID: \"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd\") " pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.830124 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.830036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-cert\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.830124 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.830088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-metrics-certs\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.830314 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.830147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwlsr\" (UniqueName: \"kubernetes.io/projected/f1a54806-fb78-4ac6-8925-9a4fd45c2443-kube-api-access-qwlsr\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.833077 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.833046    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-metrics-certs\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.833195 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.833086    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1a54806-fb78-4ac6-8925-9a4fd45c2443-cert\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.838993 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.838947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwlsr\" (UniqueName: \"kubernetes.io/projected/f1a54806-fb78-4ac6-8925-9a4fd45c2443-kube-api-access-qwlsr\") pod \"odh-model-controller-7b966d7984-lkjvs\" (UID: \"f1a54806-fb78-4ac6-8925-9a4fd45c2443\") " pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:16.852663 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.852643    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:16.875033 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:16.874996    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:17.130323 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:17.130244    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-mrntx"]
Sep 11 16:25:17.133196 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:25:17.133168    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1960cf1_3a8a_4156_ad63_1c8f0e30c0fd.slice/crio-8cd74a5f6e5b81d65469dc857ae8d005f31ca67fb28934d1803b1b5f97a60d06 WatchSource:0}: Error finding container 8cd74a5f6e5b81d65469dc857ae8d005f31ca67fb28934d1803b1b5f97a60d06: Status 404 returned error can't find the container with id 8cd74a5f6e5b81d65469dc857ae8d005f31ca67fb28934d1803b1b5f97a60d06
Sep 11 16:25:17.160629 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:17.160597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-lkjvs"]
Sep 11 16:25:17.163008 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:25:17.162951    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a54806_fb78_4ac6_8925_9a4fd45c2443.slice/crio-bc2cb3f684983b8ae9a602c0a3dbc45d0056621c1404cf5778297a79f8e82c92 WatchSource:0}: Error finding container bc2cb3f684983b8ae9a602c0a3dbc45d0056621c1404cf5778297a79f8e82c92: Status 404 returned error can't find the container with id bc2cb3f684983b8ae9a602c0a3dbc45d0056621c1404cf5778297a79f8e82c92
Sep 11 16:25:17.373557 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:17.373519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-lkjvs" event={"ID":"f1a54806-fb78-4ac6-8925-9a4fd45c2443","Type":"ContainerStarted","Data":"bc2cb3f684983b8ae9a602c0a3dbc45d0056621c1404cf5778297a79f8e82c92"}
Sep 11 16:25:17.375022 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:17.374988    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-mrntx" event={"ID":"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd","Type":"ContainerStarted","Data":"8cd74a5f6e5b81d65469dc857ae8d005f31ca67fb28934d1803b1b5f97a60d06"}
Sep 11 16:25:20.392673 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.392629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-lkjvs" event={"ID":"f1a54806-fb78-4ac6-8925-9a4fd45c2443","Type":"ContainerStarted","Data":"4d961d0cb812ea786a403e9602122a967c1f549aad8d2ce72ca5866cb1ea063d"}
Sep 11 16:25:20.393140 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.392807    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:20.394177 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.394155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-mrntx" event={"ID":"b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd","Type":"ContainerStarted","Data":"09fe7bda7431f1d26e2fe0d15be48cc6d674c0f9294fc102ab35aa0b4ce6f723"}
Sep 11 16:25:20.394302 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.394286    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:25:20.410363 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.410322    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-lkjvs" podStartSLOduration=1.595285168 podStartE2EDuration="4.410307967s" totalImagesPullingTime="2.815022799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:25:20.40930935 +0000 UTC m=+482.824792325" watchObservedRunningTime="2026-09-11 16:25:20.410307967 +0000 UTC m=+482.825790940"
Sep 11 16:25:20.427613 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:20.427564    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-mrntx" podStartSLOduration=1.6349165 podStartE2EDuration="4.427545891s" totalImagesPullingTime="2.792629391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:25:20.426608259 +0000 UTC m=+482.842091241" watchObservedRunningTime="2026-09-11 16:25:20.427545891 +0000 UTC m=+482.843028871"
Sep 11 16:25:31.401161 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:31.401130    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-lkjvs"
Sep 11 16:25:31.402814 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:25:31.402796    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-mrntx"
Sep 11 16:27:01.849376 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.849345    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:27:01.853214 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.853192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:01.855769 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.855746    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 11 16:27:01.856846 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.856721    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8975a-predictor-serving-cert\""
Sep 11 16:27:01.856846 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.856779    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8975a-kube-rbac-proxy-sar-config\""
Sep 11 16:27:01.856846 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.856810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mmq9q\""
Sep 11 16:27:01.857374 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.857349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 11 16:27:01.863982 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.863940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:27:01.917881 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.917857    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:01.918024 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.917987    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:01.918079 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:01.918039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgplj\" (UniqueName: \"kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.018665 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.018636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.018799 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.018680    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgplj\" (UniqueName: \"kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.018799 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.018719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.018799 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:27:02.018786    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8975a-predictor-serving-cert: secret "error-404-isvc-8975a-predictor-serving-cert" not found
Sep 11 16:27:02.018930 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:27:02.018877    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls podName:468bcf17-fbfc-488e-b25f-73bfb2fd3077 nodeName:}" failed. No retries permitted until 2026-09-11 16:27:02.518859863 +0000 UTC m=+584.934342827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls") pod "error-404-isvc-8975a-predictor-748969fd7d-k9kmm" (UID: "468bcf17-fbfc-488e-b25f-73bfb2fd3077") : secret "error-404-isvc-8975a-predictor-serving-cert" not found
Sep 11 16:27:02.019436 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.019408    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.030467 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.030441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgplj\" (UniqueName: \"kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.522076 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:02.522048    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:02.522227 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:27:02.522190    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8975a-predictor-serving-cert: secret "error-404-isvc-8975a-predictor-serving-cert" not found
Sep 11 16:27:02.522275 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:27:02.522261    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls podName:468bcf17-fbfc-488e-b25f-73bfb2fd3077 nodeName:}" failed. No retries permitted until 2026-09-11 16:27:03.52224746 +0000 UTC m=+585.937730419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls") pod "error-404-isvc-8975a-predictor-748969fd7d-k9kmm" (UID: "468bcf17-fbfc-488e-b25f-73bfb2fd3077") : secret "error-404-isvc-8975a-predictor-serving-cert" not found
Sep 11 16:27:03.530242 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:03.530207    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:03.532648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:03.532624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"error-404-isvc-8975a-predictor-748969fd7d-k9kmm\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") " pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:03.665479 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:03.665452    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:03.934545 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:03.934524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:27:03.936476 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:27:03.936448    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468bcf17_fbfc_488e_b25f_73bfb2fd3077.slice/crio-3c54f192b1d840f99f6564aecf3b84f5e52c20887ba6ac85db11e57c4bf3950c WatchSource:0}: Error finding container 3c54f192b1d840f99f6564aecf3b84f5e52c20887ba6ac85db11e57c4bf3950c: Status 404 returned error can't find the container with id 3c54f192b1d840f99f6564aecf3b84f5e52c20887ba6ac85db11e57c4bf3950c
Sep 11 16:27:04.893657 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:04.893616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerStarted","Data":"3c54f192b1d840f99f6564aecf3b84f5e52c20887ba6ac85db11e57c4bf3950c"}
Sep 11 16:27:15.960552 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:15.960477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerStarted","Data":"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"}
Sep 11 16:27:18.637823 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.637696    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:27:18.652388 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.638024    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:27:18.980169 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.980084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerStarted","Data":"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"}
Sep 11 16:27:18.980330 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.980188    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:18.980330 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.980299    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:18.981795 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:18.981766    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:27:19.001667 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:19.001612    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podStartSLOduration=4.096440103 podStartE2EDuration="18.001600203s" totalImagesPullingTime="13.9051601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:27:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:27:19.000135577 +0000 UTC m=+601.415618583" watchObservedRunningTime="2026-09-11 16:27:19.001600203 +0000 UTC m=+601.417083183"
Sep 11 16:27:19.984916 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:19.984873    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:27:24.989756 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:24.989725    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:27:24.990186 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:24.990160    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:27:34.990839 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:34.990797    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:27:44.990914 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:44.990833    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:27:54.990501 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:27:54.990462    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 11 16:28:04.991166 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:04.991126    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:28:45.998615 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:45.998579    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:28:45.999193 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:45.998809    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container" containerID="cri-o://79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab" gracePeriod=30
Sep 11 16:28:45.999193 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:45.998846    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kube-rbac-proxy" containerID="cri-o://be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8" gracePeriod=30
Sep 11 16:28:46.116022 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.115989    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:28:46.121648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.121577    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.124016 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.123996    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9c92a-predictor-serving-cert\""
Sep 11 16:28:46.124435 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.124414    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9c92a-kube-rbac-proxy-sar-config\""
Sep 11 16:28:46.128883 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.128861    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:28:46.157386 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.157354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.157530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.157404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.157530 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.157482    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nthrg\" (UniqueName: \"kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.258020 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.257917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.258184 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.258020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nthrg\" (UniqueName: \"kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.258184 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.258061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.258184 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:28:46.258171    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-serving-cert: secret "error-404-isvc-9c92a-predictor-serving-cert" not found
Sep 11 16:28:46.258294 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:28:46.258237    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls podName:5fd0e389-c715-4715-b7f3-ae4821860541 nodeName:}" failed. No retries permitted until 2026-09-11 16:28:46.758216558 +0000 UTC m=+689.173699521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls") pod "error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" (UID: "5fd0e389-c715-4715-b7f3-ae4821860541") : secret "error-404-isvc-9c92a-predictor-serving-cert" not found
Sep 11 16:28:46.258695 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.258676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.268653 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.268629    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nthrg\" (UniqueName: \"kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.392173 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.392139    2566 generic.go:356] "Generic (PLEG): container finished" podID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerID="be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8" exitCode=2
Sep 11 16:28:46.392337 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.392194    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerDied","Data":"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"}
Sep 11 16:28:46.762174 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.762136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:46.764583 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:46.764554    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") pod \"error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") " pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:47.033933 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:47.033878    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:47.307306 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:47.307281    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:28:47.309353 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:28:47.309326    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd0e389_c715_4715_b7f3_ae4821860541.slice/crio-441d9b68195ef3d2e49a29be8d9a54157d0326060af196619bf80bc8205e5e3e WatchSource:0}: Error finding container 441d9b68195ef3d2e49a29be8d9a54157d0326060af196619bf80bc8205e5e3e: Status 404 returned error can't find the container with id 441d9b68195ef3d2e49a29be8d9a54157d0326060af196619bf80bc8205e5e3e
Sep 11 16:28:47.398421 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:47.398395    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerStarted","Data":"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"}
Sep 11 16:28:47.398510 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:47.398450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerStarted","Data":"441d9b68195ef3d2e49a29be8d9a54157d0326060af196619bf80bc8205e5e3e"}
Sep 11 16:28:48.405378 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:48.405342    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerStarted","Data":"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"}
Sep 11 16:28:48.405809 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:48.405409    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:48.405809 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:48.405432    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:48.407002 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:48.406953    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:28:48.424276 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:48.424220    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podStartSLOduration=2.424190797 podStartE2EDuration="2.424190797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:28:48.422689673 +0000 UTC m=+690.838172671" watchObservedRunningTime="2026-09-11 16:28:48.424190797 +0000 UTC m=+690.839673778"
Sep 11 16:28:49.155544 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.155518    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:28:49.294326 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.294295    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") pod \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") "
Sep 11 16:28:49.294528 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.294337    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config\") pod \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") "
Sep 11 16:28:49.294528 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.294359    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgplj\" (UniqueName: \"kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj\") pod \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\" (UID: \"468bcf17-fbfc-488e-b25f-73bfb2fd3077\") "
Sep 11 16:28:49.294747 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.294720    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "468bcf17-fbfc-488e-b25f-73bfb2fd3077" (UID: "468bcf17-fbfc-488e-b25f-73bfb2fd3077"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:28:49.296527 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.296506    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj" (OuterVolumeSpecName: "kube-api-access-cgplj") pod "468bcf17-fbfc-488e-b25f-73bfb2fd3077" (UID: "468bcf17-fbfc-488e-b25f-73bfb2fd3077"). InnerVolumeSpecName "kube-api-access-cgplj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:28:49.296618 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.296598    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "468bcf17-fbfc-488e-b25f-73bfb2fd3077" (UID: "468bcf17-fbfc-488e-b25f-73bfb2fd3077"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:28:49.395008 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.394947    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/468bcf17-fbfc-488e-b25f-73bfb2fd3077-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:28:49.395008 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.395003    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:28:49.395008 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.395013    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgplj\" (UniqueName: \"kubernetes.io/projected/468bcf17-fbfc-488e-b25f-73bfb2fd3077-kube-api-access-cgplj\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:28:49.412239 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412204    2566 generic.go:356] "Generic (PLEG): container finished" podID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerID="79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab" exitCode=0
Sep 11 16:28:49.412701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412267    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"
Sep 11 16:28:49.412701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerDied","Data":"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"}
Sep 11 16:28:49.412701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm" event={"ID":"468bcf17-fbfc-488e-b25f-73bfb2fd3077","Type":"ContainerDied","Data":"3c54f192b1d840f99f6564aecf3b84f5e52c20887ba6ac85db11e57c4bf3950c"}
Sep 11 16:28:49.412701 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412384    2566 scope.go:117] "RemoveContainer" containerID="be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"
Sep 11 16:28:49.412923 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.412839    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:28:49.421630 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.421612    2566 scope.go:117] "RemoveContainer" containerID="79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"
Sep 11 16:28:49.431335 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.431313    2566 scope.go:117] "RemoveContainer" containerID="be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"
Sep 11 16:28:49.431669 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:28:49.431641    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8\": container with ID starting with be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8 not found: ID does not exist" containerID="be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"
Sep 11 16:28:49.431737 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.431689    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8"} err="failed to get container status \"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8\": rpc error: code = NotFound desc = could not find container \"be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8\": container with ID starting with be0ee940e3db99555e9f4f33a3a3a75253b29a9c30ba820016e2d8b4009cf1c8 not found: ID does not exist"
Sep 11 16:28:49.431737 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.431709    2566 scope.go:117] "RemoveContainer" containerID="79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"
Sep 11 16:28:49.432544 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:28:49.432394    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab\": container with ID starting with 79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab not found: ID does not exist" containerID="79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"
Sep 11 16:28:49.432544 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.432430    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab"} err="failed to get container status \"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab\": rpc error: code = NotFound desc = could not find container \"79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab\": container with ID starting with 79aff9ad9473987889dcbde05cf98017e1bff2c11062cb72dc27925a601157ab not found: ID does not exist"
Sep 11 16:28:49.435099 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.435074    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:28:49.437911 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:49.437888    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8975a-predictor-748969fd7d-k9kmm"]
Sep 11 16:28:50.111543 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:50.111509    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" path="/var/lib/kubelet/pods/468bcf17-fbfc-488e-b25f-73bfb2fd3077/volumes"
Sep 11 16:28:54.417618 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:54.417588    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:28:54.418091 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:28:54.418037    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:29:04.418054 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:04.418012    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:29:14.418383 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:14.418300    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:29:21.878355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878315    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:29:21.878727 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878690    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container"
Sep 11 16:29:21.878727 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878702    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container"
Sep 11 16:29:21.878727 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878722    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kube-rbac-proxy"
Sep 11 16:29:21.878727 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878728    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kube-rbac-proxy"
Sep 11 16:29:21.878856 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878785    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kube-rbac-proxy"
Sep 11 16:29:21.878856 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.878795    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="468bcf17-fbfc-488e-b25f-73bfb2fd3077" containerName="kserve-container"
Sep 11 16:29:21.883378 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.883355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:21.885818 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.885794    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ff919-predictor-serving-cert\""
Sep 11 16:29:21.886055 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.886036    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ff919-kube-rbac-proxy-sar-config\""
Sep 11 16:29:21.892055 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:21.892036    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:29:22.072109 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.072081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5lsq\" (UniqueName: \"kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.072275 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.072128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.072275 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.072166    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.172525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.172456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5lsq\" (UniqueName: \"kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.172525 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.172506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.172708 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.172559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.173183 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.173161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.174879 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.174860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.180589 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.180571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5lsq\" (UniqueName: \"kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq\") pod \"error-404-isvc-ff919-predictor-754fd47c76-zdg46\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") " pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.194428 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.194404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:22.463661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.463583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:29:22.467407 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:29:22.467383    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54c0a42_0d7b_4cee_9be8_154bb8d28ef9.slice/crio-76d94394f9fbc40bdb8cdab45e5719539ed2d2c3ec93dd97ba22460452d5632f WatchSource:0}: Error finding container 76d94394f9fbc40bdb8cdab45e5719539ed2d2c3ec93dd97ba22460452d5632f: Status 404 returned error can't find the container with id 76d94394f9fbc40bdb8cdab45e5719539ed2d2c3ec93dd97ba22460452d5632f
Sep 11 16:29:22.572104 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.572077    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerStarted","Data":"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"}
Sep 11 16:29:22.572188 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:22.572133    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerStarted","Data":"76d94394f9fbc40bdb8cdab45e5719539ed2d2c3ec93dd97ba22460452d5632f"}
Sep 11 16:29:23.582001 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:23.581954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerStarted","Data":"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"}
Sep 11 16:29:23.582407 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:23.582079    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:23.601155 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:23.601113    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podStartSLOduration=2.601100201 podStartE2EDuration="2.601100201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:23.598349026 +0000 UTC m=+726.013832007" watchObservedRunningTime="2026-09-11 16:29:23.601100201 +0000 UTC m=+726.016583178"
Sep 11 16:29:24.419054 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:24.419016    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 11 16:29:24.587042 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:24.587013    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:24.588486 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:24.588461    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:29:25.591736 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:25.591699    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:29:30.596295 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:30.596260    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:29:30.596874 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:30.596848    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:29:34.419148 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:34.419113    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:29:40.597074 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:40.597033    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:29:50.597522 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:29:50.597480    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:30:00.597218 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:30:00.597175    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 11 16:30:10.597822 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:30:10.597793    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:32:18.671582 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:32:18.671481    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:32:18.671582 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:32:18.671536    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:37:18.704301 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:37:18.704193    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:37:18.708259 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:37:18.704333    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:38:00.847433 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:00.847399    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:38:00.847942 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:00.847606    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container" containerID="cri-o://d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618" gracePeriod=30
Sep 11 16:38:00.847942 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:00.847640    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kube-rbac-proxy" containerID="cri-o://ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7" gracePeriod=30
Sep 11 16:38:01.000517 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.000485    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:38:01.004021 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.004000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.007037 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.007012    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a9581-kube-rbac-proxy-sar-config\""
Sep 11 16:38:01.007197 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.007179    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a9581-predictor-serving-cert\""
Sep 11 16:38:01.026724 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.026690    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:38:01.118747 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.118667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmpd6\" (UniqueName: \"kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.118747 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.118736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.118931 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.118874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.131100 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.131055    2566 generic.go:356] "Generic (PLEG): container finished" podID="5fd0e389-c715-4715-b7f3-ae4821860541" containerID="ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7" exitCode=2
Sep 11 16:38:01.131100 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.131096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerDied","Data":"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"}
Sep 11 16:38:01.219695 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.219649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.219896 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.219709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmpd6\" (UniqueName: \"kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.219896 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.219746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.220466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.220442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.222183 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.222164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.229067 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.229027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmpd6\" (UniqueName: \"kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6\") pod \"error-404-isvc-a9581-predictor-74b4589474-bz7h6\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") " pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.314542 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.314504    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:01.587679 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:01.587647    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:38:01.589902 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:38:01.589872    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c9a3e5_4152_48bb_97bf_dbca21b261fa.slice/crio-3a41cda34dbe78135ae4a29e2e6e8e1424346d22066f0f6e7f1dad9cf24df731 WatchSource:0}: Error finding container 3a41cda34dbe78135ae4a29e2e6e8e1424346d22066f0f6e7f1dad9cf24df731: Status 404 returned error can't find the container with id 3a41cda34dbe78135ae4a29e2e6e8e1424346d22066f0f6e7f1dad9cf24df731
Sep 11 16:38:02.138164 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.138128    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerStarted","Data":"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"}
Sep 11 16:38:02.138620 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.138186    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:02.138620 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.138206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerStarted","Data":"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"}
Sep 11 16:38:02.138620 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.138227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerStarted","Data":"3a41cda34dbe78135ae4a29e2e6e8e1424346d22066f0f6e7f1dad9cf24df731"}
Sep 11 16:38:02.138620 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.138246    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:02.139953 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.139926    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:02.155862 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:02.155811    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podStartSLOduration=2.155796639 podStartE2EDuration="2.155796639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:38:02.154671628 +0000 UTC m=+1244.570154610" watchObservedRunningTime="2026-09-11 16:38:02.155796639 +0000 UTC m=+1244.571279679"
Sep 11 16:38:03.143407 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:03.143366    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:04.004297 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.004271    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:38:04.044475 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.044442    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nthrg\" (UniqueName: \"kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg\") pod \"5fd0e389-c715-4715-b7f3-ae4821860541\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") "
Sep 11 16:38:04.044648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.044522    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") pod \"5fd0e389-c715-4715-b7f3-ae4821860541\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") "
Sep 11 16:38:04.044648 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.044574    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config\") pod \"5fd0e389-c715-4715-b7f3-ae4821860541\" (UID: \"5fd0e389-c715-4715-b7f3-ae4821860541\") "
Sep 11 16:38:04.045060 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.045031    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5fd0e389-c715-4715-b7f3-ae4821860541" (UID: "5fd0e389-c715-4715-b7f3-ae4821860541"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:38:04.046726 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.046699    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5fd0e389-c715-4715-b7f3-ae4821860541" (UID: "5fd0e389-c715-4715-b7f3-ae4821860541"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:38:04.046726 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.046713    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg" (OuterVolumeSpecName: "kube-api-access-nthrg") pod "5fd0e389-c715-4715-b7f3-ae4821860541" (UID: "5fd0e389-c715-4715-b7f3-ae4821860541"). InnerVolumeSpecName "kube-api-access-nthrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:38:04.145471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.145439    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nthrg\" (UniqueName: \"kubernetes.io/projected/5fd0e389-c715-4715-b7f3-ae4821860541-kube-api-access-nthrg\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:04.145471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.145467    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5fd0e389-c715-4715-b7f3-ae4821860541-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:04.145978 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.145484    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5fd0e389-c715-4715-b7f3-ae4821860541-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:04.150202 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.150173    2566 generic.go:356] "Generic (PLEG): container finished" podID="5fd0e389-c715-4715-b7f3-ae4821860541" containerID="d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618" exitCode=0
Sep 11 16:38:04.150316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.150225    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"
Sep 11 16:38:04.150316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.150252    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerDied","Data":"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"}
Sep 11 16:38:04.150316 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.150307    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6" event={"ID":"5fd0e389-c715-4715-b7f3-ae4821860541","Type":"ContainerDied","Data":"441d9b68195ef3d2e49a29be8d9a54157d0326060af196619bf80bc8205e5e3e"}
Sep 11 16:38:04.150417 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.150335    2566 scope.go:117] "RemoveContainer" containerID="ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"
Sep 11 16:38:04.159343 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.159324    2566 scope.go:117] "RemoveContainer" containerID="d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"
Sep 11 16:38:04.167735 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.167644    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:38:04.167788 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.167764    2566 scope.go:117] "RemoveContainer" containerID="ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"
Sep 11 16:38:04.168118 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:04.168095    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7\": container with ID starting with ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7 not found: ID does not exist" containerID="ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"
Sep 11 16:38:04.168208 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.168124    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7"} err="failed to get container status \"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7\": rpc error: code = NotFound desc = could not find container \"ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7\": container with ID starting with ec442b90abf520b1e232dbcb95668c216ba3b6b0b72610ba0ba2e7cf67faecc7 not found: ID does not exist"
Sep 11 16:38:04.168208 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.168144    2566 scope.go:117] "RemoveContainer" containerID="d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"
Sep 11 16:38:04.168437 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:04.168418    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618\": container with ID starting with d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618 not found: ID does not exist" containerID="d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"
Sep 11 16:38:04.168481 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.168444    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618"} err="failed to get container status \"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618\": rpc error: code = NotFound desc = could not find container \"d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618\": container with ID starting with d30659e3a3ca5e2c5f09e187e1045c41abee049d6081bbf0c22d74c946258618 not found: ID does not exist"
Sep 11 16:38:04.173641 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:04.173620    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9c92a-predictor-7cc7d7578-x4tw6"]
Sep 11 16:38:06.112220 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:06.112184    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" path="/var/lib/kubelet/pods/5fd0e389-c715-4715-b7f3-ae4821860541/volumes"
Sep 11 16:38:08.148231 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:08.148199    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:08.148693 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:08.148630    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:18.149082 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:18.149040    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:28.149594 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:28.149555    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:36.756640 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.756603    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:38:36.757105 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.756879    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container" containerID="cri-o://9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d" gracePeriod=30
Sep 11 16:38:36.757105 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.756919    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kube-rbac-proxy" containerID="cri-o://558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444" gracePeriod=30
Sep 11 16:38:36.843379 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843346    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:38:36.843815 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843798    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kube-rbac-proxy"
Sep 11 16:38:36.843815 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843815    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kube-rbac-proxy"
Sep 11 16:38:36.843939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843839    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container"
Sep 11 16:38:36.843939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843845    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container"
Sep 11 16:38:36.843939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843908    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kserve-container"
Sep 11 16:38:36.843939 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.843921    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd0e389-c715-4715-b7f3-ae4821860541" containerName="kube-rbac-proxy"
Sep 11 16:38:36.848389 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.848371    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:36.851057 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.851031    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e5408-predictor-serving-cert\""
Sep 11 16:38:36.851136 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.851065    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e5408-kube-rbac-proxy-sar-config\""
Sep 11 16:38:36.857500 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.857479    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:38:36.927769 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.927712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:36.927889 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.927853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:36.927951 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:36.927916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t84gt\" (UniqueName: \"kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.028841 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.028815    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.028981 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.028879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.028981 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.028935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t84gt\" (UniqueName: \"kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.029104 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:37.028985    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-e5408-predictor-serving-cert: secret "error-404-isvc-e5408-predictor-serving-cert" not found
Sep 11 16:38:37.029104 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:37.029065    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls podName:fe5ca859-4a2a-4551-add1-58c67ffbfd7e nodeName:}" failed. No retries permitted until 2026-09-11 16:38:37.529044783 +0000 UTC m=+1279.944527761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls") pod "error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" (UID: "fe5ca859-4a2a-4551-add1-58c67ffbfd7e") : secret "error-404-isvc-e5408-predictor-serving-cert" not found
Sep 11 16:38:37.029469 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.029450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.037606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.037586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t84gt\" (UniqueName: \"kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.314682 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.314606    2566 generic.go:356] "Generic (PLEG): container finished" podID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerID="558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444" exitCode=2
Sep 11 16:38:37.314813 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.314674    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerDied","Data":"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"}
Sep 11 16:38:37.533198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.533164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.535607 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.535579    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") pod \"error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") " pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:37.759843 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:37.759750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:38.032734 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.032703    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:38:38.034712 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:38:38.034687    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5ca859_4a2a_4551_add1_58c67ffbfd7e.slice/crio-6ae776207b79fcdb7cc683086b66db9c1f0221ed1b334ad81f0d542e0d07d90e WatchSource:0}: Error finding container 6ae776207b79fcdb7cc683086b66db9c1f0221ed1b334ad81f0d542e0d07d90e: Status 404 returned error can't find the container with id 6ae776207b79fcdb7cc683086b66db9c1f0221ed1b334ad81f0d542e0d07d90e
Sep 11 16:38:38.149236 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.149197    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 11 16:38:38.321992 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.321877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerStarted","Data":"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"}
Sep 11 16:38:38.321992 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.321945    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerStarted","Data":"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"}
Sep 11 16:38:38.322219 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.321995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerStarted","Data":"6ae776207b79fcdb7cc683086b66db9c1f0221ed1b334ad81f0d542e0d07d90e"}
Sep 11 16:38:38.322219 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.322034    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:38.322219 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.322151    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:38.323760 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.323731    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:38:38.342868 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:38.342805    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podStartSLOduration=2.342786628 podStartE2EDuration="2.342786628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:38:38.342285857 +0000 UTC m=+1280.757768840" watchObservedRunningTime="2026-09-11 16:38:38.342786628 +0000 UTC m=+1280.758269610"
Sep 11 16:38:39.327862 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:39.327821    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:38:40.103214 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.103189    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:38:40.158444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.158413    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls\") pod \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") "
Sep 11 16:38:40.158444 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.158446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config\") pod \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") "
Sep 11 16:38:40.158644 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.158517    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5lsq\" (UniqueName: \"kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq\") pod \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\" (UID: \"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9\") "
Sep 11 16:38:40.158828 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.158794    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" (UID: "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:38:40.159083 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.159060    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:40.161167 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.161142    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" (UID: "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:38:40.161167 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.161152    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq" (OuterVolumeSpecName: "kube-api-access-k5lsq") pod "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" (UID: "b54c0a42-0d7b-4cee-9be8-154bb8d28ef9"). InnerVolumeSpecName "kube-api-access-k5lsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:38:40.259915 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.259836    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5lsq\" (UniqueName: \"kubernetes.io/projected/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-kube-api-access-k5lsq\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:40.259915 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.259862    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:38:40.335237 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.335205    2566 generic.go:356] "Generic (PLEG): container finished" podID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerID="9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d" exitCode=0
Sep 11 16:38:40.335632 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.335248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerDied","Data":"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"}
Sep 11 16:38:40.335632 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.335266    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"
Sep 11 16:38:40.335632 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.335298    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46" event={"ID":"b54c0a42-0d7b-4cee-9be8-154bb8d28ef9","Type":"ContainerDied","Data":"76d94394f9fbc40bdb8cdab45e5719539ed2d2c3ec93dd97ba22460452d5632f"}
Sep 11 16:38:40.335632 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.335326    2566 scope.go:117] "RemoveContainer" containerID="558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"
Sep 11 16:38:40.344868 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.344850    2566 scope.go:117] "RemoveContainer" containerID="9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"
Sep 11 16:38:40.355144 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.355123    2566 scope.go:117] "RemoveContainer" containerID="558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"
Sep 11 16:38:40.355434 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:40.355403    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444\": container with ID starting with 558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444 not found: ID does not exist" containerID="558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"
Sep 11 16:38:40.355543 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.355442    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444"} err="failed to get container status \"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444\": rpc error: code = NotFound desc = could not find container \"558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444\": container with ID starting with 558cf297c5115dafb9a8e0aa6cf1c116823b8e1d3b4ad9d6b4e300380151c444 not found: ID does not exist"
Sep 11 16:38:40.355543 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.355466    2566 scope.go:117] "RemoveContainer" containerID="9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"
Sep 11 16:38:40.355914 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:38:40.355884    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d\": container with ID starting with 9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d not found: ID does not exist" containerID="9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"
Sep 11 16:38:40.356053 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.355922    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d"} err="failed to get container status \"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d\": rpc error: code = NotFound desc = could not find container \"9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d\": container with ID starting with 9916d7fcc3be86535bfea70799eb5a8789ebc81208d4d4e8755cb101c2d21b2d not found: ID does not exist"
Sep 11 16:38:40.357857 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.357838    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:38:40.360622 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:40.360599    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ff919-predictor-754fd47c76-zdg46"]
Sep 11 16:38:42.112611 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:42.112576    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" path="/var/lib/kubelet/pods/b54c0a42-0d7b-4cee-9be8-154bb8d28ef9/volumes"
Sep 11 16:38:44.333541 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:44.333499    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:38:44.334130 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:44.334102    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:38:48.149931 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:48.149898    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:38:54.334695 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:38:54.334652    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:39:04.334275 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:04.334234    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:39:11.076515 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.076479    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:39:11.077017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.076786    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container" containerID="cri-o://900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1" gracePeriod=30
Sep 11 16:39:11.077017 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.076795    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kube-rbac-proxy" containerID="cri-o://9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7" gracePeriod=30
Sep 11 16:39:11.236437 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236400    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:39:11.236830 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236817    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kube-rbac-proxy"
Sep 11 16:39:11.236873 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236832    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kube-rbac-proxy"
Sep 11 16:39:11.236873 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236842    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container"
Sep 11 16:39:11.236873 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236847    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container"
Sep 11 16:39:11.236980 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236918    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kube-rbac-proxy"
Sep 11 16:39:11.236980 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.236930    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b54c0a42-0d7b-4cee-9be8-154bb8d28ef9" containerName="kserve-container"
Sep 11 16:39:11.240159 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.240139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.242548 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.242524    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1ab9e-predictor-serving-cert\""
Sep 11 16:39:11.243121 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.243095    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1ab9e-kube-rbac-proxy-sar-config\""
Sep 11 16:39:11.249716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.249695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:39:11.334188 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.334108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.334355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.334286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxlw9\" (UniqueName: \"kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.334355 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.334328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.435328 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.435290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxlw9\" (UniqueName: \"kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.435328 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.435333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.435859 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.435360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.436029 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:11.435998    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-serving-cert: secret "error-404-isvc-1ab9e-predictor-serving-cert" not found
Sep 11 16:39:11.436148 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:11.436134    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls podName:40e272de-bdc9-468b-bf8f-9e7d706abc0e nodeName:}" failed. No retries permitted until 2026-09-11 16:39:11.936105967 +0000 UTC m=+1314.351588930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls") pod "error-404-isvc-1ab9e-predictor-d4c755658-thbsc" (UID: "40e272de-bdc9-468b-bf8f-9e7d706abc0e") : secret "error-404-isvc-1ab9e-predictor-serving-cert" not found
Sep 11 16:39:11.436761 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.436709    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.447678 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.447648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxlw9\" (UniqueName: \"kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.493172 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.493136    2566 generic.go:356] "Generic (PLEG): container finished" podID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerID="9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7" exitCode=2
Sep 11 16:39:11.493312 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.493196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerDied","Data":"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"}
Sep 11 16:39:11.939533 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.939496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:11.941934 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:11.941908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") pod \"error-404-isvc-1ab9e-predictor-d4c755658-thbsc\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") " pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:12.151888 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:12.151852    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:12.426745 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:12.426698    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:39:12.429020 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:39:12.428987    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e272de_bdc9_468b_bf8f_9e7d706abc0e.slice/crio-71960be62f97d670f819c41e71eb69cc4045a02b854cb2122ac0ef407bfd7f76 WatchSource:0}: Error finding container 71960be62f97d670f819c41e71eb69cc4045a02b854cb2122ac0ef407bfd7f76: Status 404 returned error can't find the container with id 71960be62f97d670f819c41e71eb69cc4045a02b854cb2122ac0ef407bfd7f76
Sep 11 16:39:12.498982 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:12.498937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerStarted","Data":"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"}
Sep 11 16:39:12.499086 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:12.499004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerStarted","Data":"71960be62f97d670f819c41e71eb69cc4045a02b854cb2122ac0ef407bfd7f76"}
Sep 11 16:39:13.144461 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.144421    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Sep 11 16:39:13.505565 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.505475    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerStarted","Data":"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"}
Sep 11 16:39:13.505565 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.505541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:13.505565 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.505562    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:13.507132 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.507104    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:13.524167 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:13.524100    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podStartSLOduration=2.524078163 podStartE2EDuration="2.524078163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:13.522437246 +0000 UTC m=+1315.937920228" watchObservedRunningTime="2026-09-11 16:39:13.524078163 +0000 UTC m=+1315.939561149"
Sep 11 16:39:14.331562 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.331523    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:39:14.334796 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.334767    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 11 16:39:14.463864 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.463783    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmpd6\" (UniqueName: \"kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6\") pod \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") "
Sep 11 16:39:14.463864 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.463833    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config\") pod \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") "
Sep 11 16:39:14.463864 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.463856    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls\") pod \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\" (UID: \"d3c9a3e5-4152-48bb-97bf-dbca21b261fa\") "
Sep 11 16:39:14.464322 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.464292    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d3c9a3e5-4152-48bb-97bf-dbca21b261fa" (UID: "d3c9a3e5-4152-48bb-97bf-dbca21b261fa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:39:14.466019 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.465993    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6" (OuterVolumeSpecName: "kube-api-access-bmpd6") pod "d3c9a3e5-4152-48bb-97bf-dbca21b261fa" (UID: "d3c9a3e5-4152-48bb-97bf-dbca21b261fa"). InnerVolumeSpecName "kube-api-access-bmpd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:39:14.466019 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.465993    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d3c9a3e5-4152-48bb-97bf-dbca21b261fa" (UID: "d3c9a3e5-4152-48bb-97bf-dbca21b261fa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:39:14.512808 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.512773    2566 generic.go:356] "Generic (PLEG): container finished" podID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerID="900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1" exitCode=0
Sep 11 16:39:14.513283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.512821    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"
Sep 11 16:39:14.513283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.512861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerDied","Data":"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"}
Sep 11 16:39:14.513283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.512938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6" event={"ID":"d3c9a3e5-4152-48bb-97bf-dbca21b261fa","Type":"ContainerDied","Data":"3a41cda34dbe78135ae4a29e2e6e8e1424346d22066f0f6e7f1dad9cf24df731"}
Sep 11 16:39:14.513283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.513002    2566 scope.go:117] "RemoveContainer" containerID="9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"
Sep 11 16:39:14.513488 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.513390    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:14.528475 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.528455    2566 scope.go:117] "RemoveContainer" containerID="900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"
Sep 11 16:39:14.536856 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.536830    2566 scope.go:117] "RemoveContainer" containerID="9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"
Sep 11 16:39:14.537196 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:14.537176    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7\": container with ID starting with 9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7 not found: ID does not exist" containerID="9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"
Sep 11 16:39:14.537267 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.537206    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7"} err="failed to get container status \"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7\": rpc error: code = NotFound desc = could not find container \"9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7\": container with ID starting with 9e5353be9f54210ea977a27429a63d8bd8f9ca965b3c6b2439e2d5ec6f0b51b7 not found: ID does not exist"
Sep 11 16:39:14.537267 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.537224    2566 scope.go:117] "RemoveContainer" containerID="900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"
Sep 11 16:39:14.537518 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:14.537501    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1\": container with ID starting with 900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1 not found: ID does not exist" containerID="900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"
Sep 11 16:39:14.537583 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.537524    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1"} err="failed to get container status \"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1\": rpc error: code = NotFound desc = could not find container \"900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1\": container with ID starting with 900e10284648f2da1080c3d195ffb706b55d0e6b6218ab747d059a3baa49bac1 not found: ID does not exist"
Sep 11 16:39:14.541657 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.541635    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:39:14.546051 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.546027    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a9581-predictor-74b4589474-bz7h6"]
Sep 11 16:39:14.565420 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.565397    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:14.565492 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.565420    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:14.565492 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:14.565433    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmpd6\" (UniqueName: \"kubernetes.io/projected/d3c9a3e5-4152-48bb-97bf-dbca21b261fa-kube-api-access-bmpd6\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:16.112049 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:16.112012    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" path="/var/lib/kubelet/pods/d3c9a3e5-4152-48bb-97bf-dbca21b261fa/volumes"
Sep 11 16:39:19.517549 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:19.517515    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:39:19.518014 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:19.517989    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:24.335931 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:24.335901    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:39:29.518496 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:29.518452    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:39.518719 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:39.518673    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:46.892286 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:46.892191    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:39:46.892712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:46.892487    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container" containerID="cri-o://d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2" gracePeriod=30
Sep 11 16:39:46.892712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:46.892536    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kube-rbac-proxy" containerID="cri-o://5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531" gracePeriod=30
Sep 11 16:39:47.098424 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098387    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:39:47.098784 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098772    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kube-rbac-proxy"
Sep 11 16:39:47.098831 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098786    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kube-rbac-proxy"
Sep 11 16:39:47.098831 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098811    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container"
Sep 11 16:39:47.098831 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098816    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container"
Sep 11 16:39:47.098924 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098878    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kserve-container"
Sep 11 16:39:47.098924 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.098887    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c9a3e5-4152-48bb-97bf-dbca21b261fa" containerName="kube-rbac-proxy"
Sep 11 16:39:47.101493 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.101473    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.104039 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.104009    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ea3c5-kube-rbac-proxy-sar-config\""
Sep 11 16:39:47.104443 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.104422    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ea3c5-predictor-serving-cert\""
Sep 11 16:39:47.113814 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.113781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:39:47.255009 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.254884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctdhr\" (UniqueName: \"kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.255009 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.254929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.255009 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.254993    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.355940 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.355897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctdhr\" (UniqueName: \"kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.355940 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.355954    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.356211 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.355996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.356706 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.356680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.358477 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.358450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.365262 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.365234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctdhr\" (UniqueName: \"kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr\") pod \"error-404-isvc-ea3c5-predictor-848549444-hb7jc\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") " pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.414187 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.414144    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:47.686055 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.686014    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:39:47.686198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.686056    2566 generic.go:356] "Generic (PLEG): container finished" podID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerID="5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531" exitCode=2
Sep 11 16:39:47.686198 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:47.686094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerDied","Data":"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"}
Sep 11 16:39:47.688284 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:39:47.688246    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda08659d_2d8b_4597_9167_66d3e9859aad.slice/crio-b4b48be18619e02d390dca6dd9fce0e04885ca4c2bdf70b1c42abe8176458035 WatchSource:0}: Error finding container b4b48be18619e02d390dca6dd9fce0e04885ca4c2bdf70b1c42abe8176458035: Status 404 returned error can't find the container with id b4b48be18619e02d390dca6dd9fce0e04885ca4c2bdf70b1c42abe8176458035
Sep 11 16:39:48.693988 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:48.693936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerStarted","Data":"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"}
Sep 11 16:39:48.694436 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:48.694010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerStarted","Data":"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"}
Sep 11 16:39:48.694436 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:48.694032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerStarted","Data":"b4b48be18619e02d390dca6dd9fce0e04885ca4c2bdf70b1c42abe8176458035"}
Sep 11 16:39:48.694436 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:48.694109    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:48.714088 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:48.714031    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podStartSLOduration=1.714014641 podStartE2EDuration="1.714014641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:48.711381407 +0000 UTC m=+1351.126864389" watchObservedRunningTime="2026-09-11 16:39:48.714014641 +0000 UTC m=+1351.129497622"
Sep 11 16:39:49.329033 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:49.328950    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Sep 11 16:39:49.518190 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:49.518150    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:39:49.699430 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:49.699342    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:49.700850 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:49.700822    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:39:50.454441 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.454412    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:39:50.586286 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.586250    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config\") pod \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") "
Sep 11 16:39:50.586497 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.586300    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") pod \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") "
Sep 11 16:39:50.586497 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.586320    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t84gt\" (UniqueName: \"kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt\") pod \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\" (UID: \"fe5ca859-4a2a-4551-add1-58c67ffbfd7e\") "
Sep 11 16:39:50.586713 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.586688    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fe5ca859-4a2a-4551-add1-58c67ffbfd7e" (UID: "fe5ca859-4a2a-4551-add1-58c67ffbfd7e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:39:50.588522 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.588498    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fe5ca859-4a2a-4551-add1-58c67ffbfd7e" (UID: "fe5ca859-4a2a-4551-add1-58c67ffbfd7e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:39:50.588522 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.588509    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt" (OuterVolumeSpecName: "kube-api-access-t84gt") pod "fe5ca859-4a2a-4551-add1-58c67ffbfd7e" (UID: "fe5ca859-4a2a-4551-add1-58c67ffbfd7e"). InnerVolumeSpecName "kube-api-access-t84gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:39:50.687661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.687617    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.687661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.687655    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t84gt\" (UniqueName: \"kubernetes.io/projected/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-api-access-t84gt\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.687661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.687670    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe5ca859-4a2a-4551-add1-58c67ffbfd7e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.705789 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.705752    2566 generic.go:356] "Generic (PLEG): container finished" podID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerID="d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2" exitCode=0
Sep 11 16:39:50.706279 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.705802    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"
Sep 11 16:39:50.706279 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.705845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerDied","Data":"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"}
Sep 11 16:39:50.706279 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.705924    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6" event={"ID":"fe5ca859-4a2a-4551-add1-58c67ffbfd7e","Type":"ContainerDied","Data":"6ae776207b79fcdb7cc683086b66db9c1f0221ed1b334ad81f0d542e0d07d90e"}
Sep 11 16:39:50.706279 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.705988    2566 scope.go:117] "RemoveContainer" containerID="5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"
Sep 11 16:39:50.706473 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.706301    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:39:50.715766 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.715746    2566 scope.go:117] "RemoveContainer" containerID="d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"
Sep 11 16:39:50.724899 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.724872    2566 scope.go:117] "RemoveContainer" containerID="5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"
Sep 11 16:39:50.725372 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:50.725343    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531\": container with ID starting with 5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531 not found: ID does not exist" containerID="5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"
Sep 11 16:39:50.725471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.725384    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531"} err="failed to get container status \"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531\": rpc error: code = NotFound desc = could not find container \"5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531\": container with ID starting with 5db6a556b722c21ecaa9835d1996e83e2b352356db6c7ace49711bd8e7854531 not found: ID does not exist"
Sep 11 16:39:50.725471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.725412    2566 scope.go:117] "RemoveContainer" containerID="d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"
Sep 11 16:39:50.725795 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:39:50.725771    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2\": container with ID starting with d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2 not found: ID does not exist" containerID="d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"
Sep 11 16:39:50.725849 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.725801    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2"} err="failed to get container status \"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2\": rpc error: code = NotFound desc = could not find container \"d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2\": container with ID starting with d9d194980db4517bee98093e309930455539fc707233b2a3f1d220b4ebeb3ff2 not found: ID does not exist"
Sep 11 16:39:50.729548 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.729497    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:39:50.734030 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:50.733993    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e5408-predictor-7cfd9ff86-4zpt6"]
Sep 11 16:39:52.112000 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:52.111940    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" path="/var/lib/kubelet/pods/fe5ca859-4a2a-4551-add1-58c67ffbfd7e/volumes"
Sep 11 16:39:55.712169 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:55.712138    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:39:55.712633 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:55.712603    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:39:59.519287 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:39:59.519251    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:40:05.712625 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:40:05.712577    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:40:15.713188 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:40:15.713142    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:40:25.713518 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:40:25.713477    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 11 16:40:35.714154 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:40:35.714122    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:42:18.737999 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:42:18.737876    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:42:18.742167 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:42:18.740930    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:47:18.770821 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:47:18.770707    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:47:18.776925 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:47:18.774517    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:48:26.060949 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.060903    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:48:26.061564 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.061358    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" containerID="cri-o://30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e" gracePeriod=30
Sep 11 16:48:26.061564 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.061406    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kube-rbac-proxy" containerID="cri-o://03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2" gracePeriod=30
Sep 11 16:48:26.153049 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153011    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:48:26.153495 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153477    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kube-rbac-proxy"
Sep 11 16:48:26.153585 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153502    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kube-rbac-proxy"
Sep 11 16:48:26.153585 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153553    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container"
Sep 11 16:48:26.153585 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153563    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container"
Sep 11 16:48:26.153743 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153652    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kube-rbac-proxy"
Sep 11 16:48:26.153743 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.153666    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5ca859-4a2a-4551-add1-58c67ffbfd7e" containerName="kserve-container"
Sep 11 16:48:26.157026 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.156995    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.159588 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.159562    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ddc34-predictor-serving-cert\""
Sep 11 16:48:26.159726 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.159590    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ddc34-kube-rbac-proxy-sar-config\""
Sep 11 16:48:26.166850 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.166816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:48:26.175213 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.175180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.175321 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.175216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frl27\" (UniqueName: \"kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.175321 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.175253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.242282 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.242246    2566 generic.go:356] "Generic (PLEG): container finished" podID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerID="03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2" exitCode=2
Sep 11 16:48:26.242486 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.242315    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerDied","Data":"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"}
Sep 11 16:48:26.275991 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.275935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.275991 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.275997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frl27\" (UniqueName: \"kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.276227 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.276100    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.276227 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:48:26.276111    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-serving-cert: secret "error-404-isvc-ddc34-predictor-serving-cert" not found
Sep 11 16:48:26.276227 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:48:26.276188    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls podName:6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5 nodeName:}" failed. No retries permitted until 2026-09-11 16:48:26.776165585 +0000 UTC m=+1869.191648549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls") pod "error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" (UID: "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5") : secret "error-404-isvc-ddc34-predictor-serving-cert" not found
Sep 11 16:48:26.276802 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.276783    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.287042 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.287006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frl27\" (UniqueName: \"kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.781574 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.781526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:26.784225 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:26.784201    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") pod \"error-404-isvc-ddc34-predictor-64558dffdb-gvgtl\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") " pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:27.069312 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:27.069212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:27.342998 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:27.342954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:48:27.345022 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:48:27.344995    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a436a79_c6a0_4ae0_bcc7_d77a3e7f7ee5.slice/crio-1b7c20e6c6c142d354323929daca89eab6a70f36939547b1340e5145f4ae6c73 WatchSource:0}: Error finding container 1b7c20e6c6c142d354323929daca89eab6a70f36939547b1340e5145f4ae6c73: Status 404 returned error can't find the container with id 1b7c20e6c6c142d354323929daca89eab6a70f36939547b1340e5145f4ae6c73
Sep 11 16:48:28.254157 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:28.254122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerStarted","Data":"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"}
Sep 11 16:48:28.254661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:28.254188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerStarted","Data":"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"}
Sep 11 16:48:28.254661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:28.254218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerStarted","Data":"1b7c20e6c6c142d354323929daca89eab6a70f36939547b1340e5145f4ae6c73"}
Sep 11 16:48:28.254661 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:28.254260    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:28.273511 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:28.273453    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podStartSLOduration=2.273434658 podStartE2EDuration="2.273434658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:28.272020804 +0000 UTC m=+1870.687503789" watchObservedRunningTime="2026-09-11 16:48:28.273434658 +0000 UTC m=+1870.688917638"
Sep 11 16:48:29.259372 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.259334    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:29.260829 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.260797    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:48:29.513929 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.513841    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Sep 11 16:48:29.524598 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.524539    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 11 16:48:29.622458 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.622431    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:48:29.707267 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.707226    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxlw9\" (UniqueName: \"kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9\") pod \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") "
Sep 11 16:48:29.707420 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.707327    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") pod \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") "
Sep 11 16:48:29.707420 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.707354    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config\") pod \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\" (UID: \"40e272de-bdc9-468b-bf8f-9e7d706abc0e\") "
Sep 11 16:48:29.707738 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.707707    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "40e272de-bdc9-468b-bf8f-9e7d706abc0e" (UID: "40e272de-bdc9-468b-bf8f-9e7d706abc0e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:48:29.709410 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.709383    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9" (OuterVolumeSpecName: "kube-api-access-gxlw9") pod "40e272de-bdc9-468b-bf8f-9e7d706abc0e" (UID: "40e272de-bdc9-468b-bf8f-9e7d706abc0e"). InnerVolumeSpecName "kube-api-access-gxlw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:29.709513 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.709390    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "40e272de-bdc9-468b-bf8f-9e7d706abc0e" (UID: "40e272de-bdc9-468b-bf8f-9e7d706abc0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:48:29.808201 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.808160    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40e272de-bdc9-468b-bf8f-9e7d706abc0e-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:48:29.808201 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.808197    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:48:29.808201 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:29.808208    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxlw9\" (UniqueName: \"kubernetes.io/projected/40e272de-bdc9-468b-bf8f-9e7d706abc0e-kube-api-access-gxlw9\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:48:30.265583 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.265548    2566 generic.go:356] "Generic (PLEG): container finished" podID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerID="30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e" exitCode=0
Sep 11 16:48:30.266066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.265614    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"
Sep 11 16:48:30.266066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.265635    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerDied","Data":"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"}
Sep 11 16:48:30.266066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.265709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc" event={"ID":"40e272de-bdc9-468b-bf8f-9e7d706abc0e","Type":"ContainerDied","Data":"71960be62f97d670f819c41e71eb69cc4045a02b854cb2122ac0ef407bfd7f76"}
Sep 11 16:48:30.266066 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.265747    2566 scope.go:117] "RemoveContainer" containerID="03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"
Sep 11 16:48:30.266283 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.266112    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:48:30.275064 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.275035    2566 scope.go:117] "RemoveContainer" containerID="30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"
Sep 11 16:48:30.284257 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.284233    2566 scope.go:117] "RemoveContainer" containerID="03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"
Sep 11 16:48:30.284606 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:48:30.284579    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2\": container with ID starting with 03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2 not found: ID does not exist" containerID="03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"
Sep 11 16:48:30.284716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.284615    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2"} err="failed to get container status \"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2\": rpc error: code = NotFound desc = could not find container \"03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2\": container with ID starting with 03a5180cd3ec4bc76eff9488d4142707b2ca7278909ed938db021a348b33f8c2 not found: ID does not exist"
Sep 11 16:48:30.284716 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.284636    2566 scope.go:117] "RemoveContainer" containerID="30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"
Sep 11 16:48:30.284940 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:48:30.284922    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e\": container with ID starting with 30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e not found: ID does not exist" containerID="30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"
Sep 11 16:48:30.285031 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.284943    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e"} err="failed to get container status \"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e\": rpc error: code = NotFound desc = could not find container \"30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e\": container with ID starting with 30285c55d32e0cd427baf64467cf292ba60d3e5126b747f0bf7619fbbb07158e not found: ID does not exist"
Sep 11 16:48:30.285987 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.285946    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:48:30.290411 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:30.290386    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ab9e-predictor-d4c755658-thbsc"]
Sep 11 16:48:32.112855 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:32.112818    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" path="/var/lib/kubelet/pods/40e272de-bdc9-468b-bf8f-9e7d706abc0e/volumes"
Sep 11 16:48:35.270497 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:35.270464    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:48:35.271018 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:35.270991    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:48:45.271372 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:45.271332    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:48:55.271798 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:48:55.271755    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:49:01.748797 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.748759    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:49:01.749366 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.749063    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container" containerID="cri-o://2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de" gracePeriod=30
Sep 11 16:49:01.749366 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.749237    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kube-rbac-proxy" containerID="cri-o://898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef" gracePeriod=30
Sep 11 16:49:01.974126 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974090    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 16:49:01.974523 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974509    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kube-rbac-proxy"
Sep 11 16:49:01.974573 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974525    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kube-rbac-proxy"
Sep 11 16:49:01.974573 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974545    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container"
Sep 11 16:49:01.974573 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974559    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container"
Sep 11 16:49:01.974674 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974623    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kserve-container"
Sep 11 16:49:01.974674 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.974630    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e272de-bdc9-468b-bf8f-9e7d706abc0e" containerName="kube-rbac-proxy"
Sep 11 16:49:01.977793 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.977776    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:01.981543 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.981520    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-482a1-kube-rbac-proxy-sar-config\""
Sep 11 16:49:01.982606 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:01.982589    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-482a1-predictor-serving-cert\""
Sep 11 16:49:02.000890 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.000815    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 16:49:02.116181 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.116144    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhkjz\" (UniqueName: \"kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.116359 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.116195    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.116359 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.116232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.217702 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.217659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhkjz\" (UniqueName: \"kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.217902 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.217731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.217902 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.217768    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.218520 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.218497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.220471 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.220446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.226310 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.226285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhkjz\" (UniqueName: \"kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz\") pod \"error-404-isvc-482a1-predictor-686985f998-wdxbb\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") " pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.288204 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.288170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:02.433887 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.433852    2566 generic.go:356] "Generic (PLEG): container finished" podID="da08659d-2d8b-4597-9167-66d3e9859aad" containerID="898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef" exitCode=2
Sep 11 16:49:02.434086 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.433920    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerDied","Data":"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"}
Sep 11 16:49:02.562997 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:02.562974    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 16:49:02.565091 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:49:02.565067    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839e821c_fa44_488b_ad3a_1754ded0cb23.slice/crio-0eb113fcec3e819c7571f61b4ea20386932957f9f78e522ba175256c4a2c133b WatchSource:0}: Error finding container 0eb113fcec3e819c7571f61b4ea20386932957f9f78e522ba175256c4a2c133b: Status 404 returned error can't find the container with id 0eb113fcec3e819c7571f61b4ea20386932957f9f78e522ba175256c4a2c133b
Sep 11 16:49:03.441307 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.441268    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerStarted","Data":"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"}
Sep 11 16:49:03.441712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.441334    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerStarted","Data":"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"}
Sep 11 16:49:03.441712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.441367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerStarted","Data":"0eb113fcec3e819c7571f61b4ea20386932957f9f78e522ba175256c4a2c133b"}
Sep 11 16:49:03.441712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.441467    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:03.441712 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.441497    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:03.442970 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.442930    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:03.462199 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:03.462143    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podStartSLOduration=2.462125964 podStartE2EDuration="2.462125964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:03.460570996 +0000 UTC m=+1905.876053979" watchObservedRunningTime="2026-09-11 16:49:03.462125964 +0000 UTC m=+1905.877608945"
Sep 11 16:49:04.446331 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:04.446282    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:05.271706 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.271670    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 11 16:49:05.314583 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.314556    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:49:05.344979 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.344933    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctdhr\" (UniqueName: \"kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr\") pod \"da08659d-2d8b-4597-9167-66d3e9859aad\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") "
Sep 11 16:49:05.345162 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.345035    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config\") pod \"da08659d-2d8b-4597-9167-66d3e9859aad\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") "
Sep 11 16:49:05.345162 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.345079    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls\") pod \"da08659d-2d8b-4597-9167-66d3e9859aad\" (UID: \"da08659d-2d8b-4597-9167-66d3e9859aad\") "
Sep 11 16:49:05.345434 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.345408    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "da08659d-2d8b-4597-9167-66d3e9859aad" (UID: "da08659d-2d8b-4597-9167-66d3e9859aad"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:49:05.347227 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.347206    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "da08659d-2d8b-4597-9167-66d3e9859aad" (UID: "da08659d-2d8b-4597-9167-66d3e9859aad"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:49:05.347315 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.347301    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr" (OuterVolumeSpecName: "kube-api-access-ctdhr") pod "da08659d-2d8b-4597-9167-66d3e9859aad" (UID: "da08659d-2d8b-4597-9167-66d3e9859aad"). InnerVolumeSpecName "kube-api-access-ctdhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:05.445996 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.445896    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da08659d-2d8b-4597-9167-66d3e9859aad-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.445996 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.445931    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da08659d-2d8b-4597-9167-66d3e9859aad-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.445996 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.445942    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctdhr\" (UniqueName: \"kubernetes.io/projected/da08659d-2d8b-4597-9167-66d3e9859aad-kube-api-access-ctdhr\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.452042 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.452011    2566 generic.go:356] "Generic (PLEG): container finished" podID="da08659d-2d8b-4597-9167-66d3e9859aad" containerID="2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de" exitCode=0
Sep 11 16:49:05.452392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.452069    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"
Sep 11 16:49:05.452392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.452100    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerDied","Data":"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"}
Sep 11 16:49:05.452392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.452161    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc" event={"ID":"da08659d-2d8b-4597-9167-66d3e9859aad","Type":"ContainerDied","Data":"b4b48be18619e02d390dca6dd9fce0e04885ca4c2bdf70b1c42abe8176458035"}
Sep 11 16:49:05.452392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.452189    2566 scope.go:117] "RemoveContainer" containerID="898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"
Sep 11 16:49:05.461266 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.461246    2566 scope.go:117] "RemoveContainer" containerID="2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"
Sep 11 16:49:05.469117 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.469101    2566 scope.go:117] "RemoveContainer" containerID="898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"
Sep 11 16:49:05.469418 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:49:05.469394    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef\": container with ID starting with 898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef not found: ID does not exist" containerID="898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"
Sep 11 16:49:05.469466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.469427    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef"} err="failed to get container status \"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef\": rpc error: code = NotFound desc = could not find container \"898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef\": container with ID starting with 898c507e3cd50cceea01cb743dac0a5e6c5486e49c6dd403210820df0b7ff0ef not found: ID does not exist"
Sep 11 16:49:05.469466 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.469444    2566 scope.go:117] "RemoveContainer" containerID="2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"
Sep 11 16:49:05.469736 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:49:05.469716    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de\": container with ID starting with 2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de not found: ID does not exist" containerID="2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"
Sep 11 16:49:05.469832 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.469741    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de"} err="failed to get container status \"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de\": rpc error: code = NotFound desc = could not find container \"2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de\": container with ID starting with 2a4e56e30ba5c658c4c80aa09a5dbd9b007c1e3942075c918652f2fb8e40f2de not found: ID does not exist"
Sep 11 16:49:05.474217 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.474193    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:49:05.477346 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:05.477326    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ea3c5-predictor-848549444-hb7jc"]
Sep 11 16:49:06.112542 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:06.112509    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" path="/var/lib/kubelet/pods/da08659d-2d8b-4597-9167-66d3e9859aad/volumes"
Sep 11 16:49:09.451764 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:09.451733    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:09.452215 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:09.452187    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:15.272133 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:15.272100    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:49:19.452780 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:19.452741    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:29.452457 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:29.452411    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:36.314884 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.314844    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:49:36.315330 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.315131    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container" containerID="cri-o://ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad" gracePeriod=30
Sep 11 16:49:36.315330 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.315161    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kube-rbac-proxy" containerID="cri-o://b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca" gracePeriod=30
Sep 11 16:49:36.502988 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.502939    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:49:36.503392 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503377    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kube-rbac-proxy"
Sep 11 16:49:36.503439 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503394    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kube-rbac-proxy"
Sep 11 16:49:36.503439 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503405    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container"
Sep 11 16:49:36.503439 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503410    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container"
Sep 11 16:49:36.503532 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503469    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kserve-container"
Sep 11 16:49:36.503532 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.503477    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da08659d-2d8b-4597-9167-66d3e9859aad" containerName="kube-rbac-proxy"
Sep 11 16:49:36.506937 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.506916    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.509203 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.509184    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-20f8f-kube-rbac-proxy-sar-config\""
Sep 11 16:49:36.509276 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.509211    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-20f8f-predictor-serving-cert\""
Sep 11 16:49:36.515083 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.515050    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:49:36.608839 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.608749    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerID="b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca" exitCode=2
Sep 11 16:49:36.608839 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.608818    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerDied","Data":"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"}
Sep 11 16:49:36.632082 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.632044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmphs\" (UniqueName: \"kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.632250 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.632110    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.632250 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.632148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.732823 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.732774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.733020 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.732924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmphs\" (UniqueName: \"kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.733020 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.732997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.733460 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.733434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.735457 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.735437    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.741295 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.741276    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmphs\" (UniqueName: \"kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs\") pod \"error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") " pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:36.817669 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:36.817641    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:37.089652 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.089630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:49:37.091251 ip-10-0-128-17 kubenswrapper[2566]: W0911 16:49:37.091223    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd766b2f0_9eec_46e8_a152_653641963131.slice/crio-d2668a010dfbd6ebaa78d70e2e6b8d45c1107ce765c569f1dde1f13a5e0d42e1 WatchSource:0}: Error finding container d2668a010dfbd6ebaa78d70e2e6b8d45c1107ce765c569f1dde1f13a5e0d42e1: Status 404 returned error can't find the container with id d2668a010dfbd6ebaa78d70e2e6b8d45c1107ce765c569f1dde1f13a5e0d42e1
Sep 11 16:49:37.615089 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.615049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerStarted","Data":"84fc9af0bb85a3fcf6f20473e61a93d953bd37907f48463ec240bb8232cfab27"}
Sep 11 16:49:37.615507 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.615165    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerStarted","Data":"cfadd07d1c98f990461d6a3f9f5a8984750d21fbeee96c0e351b318788b695e1"}
Sep 11 16:49:37.615507 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.615193    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:37.615507 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.615213    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerStarted","Data":"d2668a010dfbd6ebaa78d70e2e6b8d45c1107ce765c569f1dde1f13a5e0d42e1"}
Sep 11 16:49:37.632566 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:37.632501    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podStartSLOduration=1.632480128 podStartE2EDuration="1.632480128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:37.631301002 +0000 UTC m=+1940.046783983" watchObservedRunningTime="2026-09-11 16:49:37.632480128 +0000 UTC m=+1940.047963108"
Sep 11 16:49:38.619322 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:38.619289    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:38.620945 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:38.620912    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:49:39.453197 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.453153    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 11 16:49:39.559607 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.559584    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:49:39.625579 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625497    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerID="ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad" exitCode=0
Sep 11 16:49:39.626025 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625575    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"
Sep 11 16:49:39.626025 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerDied","Data":"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"}
Sep 11 16:49:39.626025 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl" event={"ID":"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5","Type":"ContainerDied","Data":"1b7c20e6c6c142d354323929daca89eab6a70f36939547b1340e5145f4ae6c73"}
Sep 11 16:49:39.626025 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625927    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:49:39.626304 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.625733    2566 scope.go:117] "RemoveContainer" containerID="b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"
Sep 11 16:49:39.635129 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.635090    2566 scope.go:117] "RemoveContainer" containerID="ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"
Sep 11 16:49:39.642642 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.642626    2566 scope.go:117] "RemoveContainer" containerID="b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"
Sep 11 16:49:39.642928 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:49:39.642910    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca\": container with ID starting with b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca not found: ID does not exist" containerID="b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"
Sep 11 16:49:39.643003 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.642937    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca"} err="failed to get container status \"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca\": rpc error: code = NotFound desc = could not find container \"b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca\": container with ID starting with b65de8478d3b7cd1df37835353205a979bb038fc905a46e343a3885b79b00dca not found: ID does not exist"
Sep 11 16:49:39.643003 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.642952    2566 scope.go:117] "RemoveContainer" containerID="ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"
Sep 11 16:49:39.643296 ip-10-0-128-17 kubenswrapper[2566]: E0911 16:49:39.643277    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad\": container with ID starting with ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad not found: ID does not exist" containerID="ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"
Sep 11 16:49:39.643352 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.643305    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad"} err="failed to get container status \"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad\": rpc error: code = NotFound desc = could not find container \"ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad\": container with ID starting with ddd4c87df98e7d61290bc61cd2833bdd3eea65d78f16b598f83668febaf4bbad not found: ID does not exist"
Sep 11 16:49:39.657587 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.657566    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") pod \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") "
Sep 11 16:49:39.657658 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.657609    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config\") pod \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") "
Sep 11 16:49:39.657710 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.657699    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frl27\" (UniqueName: \"kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27\") pod \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\" (UID: \"6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5\") "
Sep 11 16:49:39.657954 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.657933    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" (UID: "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:49:39.659493 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.659473    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" (UID: "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:49:39.659683 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.659662    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27" (OuterVolumeSpecName: "kube-api-access-frl27") pod "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" (UID: "6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5"). InnerVolumeSpecName "kube-api-access-frl27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:39.759339 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.759307    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:39.759339 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.759332    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frl27\" (UniqueName: \"kubernetes.io/projected/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-kube-api-access-frl27\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:39.759679 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.759360    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:49:39.951194 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.951157    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:49:39.957292 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:39.957265    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ddc34-predictor-64558dffdb-gvgtl"]
Sep 11 16:49:40.111645 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:40.111615    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" path="/var/lib/kubelet/pods/6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5/volumes"
Sep 11 16:49:44.630783 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:44.630751    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:49:44.631367 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:44.631332    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:49:49.453135 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:49.453102    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 16:49:54.631300 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:49:54.631252    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:50:04.632287 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:50:04.632238    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:50:14.631938 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:50:14.631848    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Sep 11 16:50:24.632672 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:50:24.632642    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:52:18.803926 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:52:18.803894    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:52:18.808128 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:52:18.808104    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:57:18.839633 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:57:18.839523    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:57:18.849333 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:57:18.849311    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 16:58:51.152639 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:51.152595    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:58:51.153252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:51.152889    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container" containerID="cri-o://cfadd07d1c98f990461d6a3f9f5a8984750d21fbeee96c0e351b318788b695e1" gracePeriod=30
Sep 11 16:58:51.153252 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:51.152907    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kube-rbac-proxy" containerID="cri-o://84fc9af0bb85a3fcf6f20473e61a93d953bd37907f48463ec240bb8232cfab27" gracePeriod=30
Sep 11 16:58:51.289325 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:51.289289    2566 generic.go:356] "Generic (PLEG): container finished" podID="d766b2f0-9eec-46e8-a152-653641963131" containerID="84fc9af0bb85a3fcf6f20473e61a93d953bd37907f48463ec240bb8232cfab27" exitCode=2
Sep 11 16:58:51.289498 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:51.289358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerDied","Data":"84fc9af0bb85a3fcf6f20473e61a93d953bd37907f48463ec240bb8232cfab27"}
Sep 11 16:58:54.307090 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.307057    2566 generic.go:356] "Generic (PLEG): container finished" podID="d766b2f0-9eec-46e8-a152-653641963131" containerID="cfadd07d1c98f990461d6a3f9f5a8984750d21fbeee96c0e351b318788b695e1" exitCode=0
Sep 11 16:58:54.307443 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.307127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerDied","Data":"cfadd07d1c98f990461d6a3f9f5a8984750d21fbeee96c0e351b318788b695e1"}
Sep 11 16:58:54.403844 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.403818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:58:54.497569 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.497476    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config\") pod \"d766b2f0-9eec-46e8-a152-653641963131\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") "
Sep 11 16:58:54.497569 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.497533    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmphs\" (UniqueName: \"kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs\") pod \"d766b2f0-9eec-46e8-a152-653641963131\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") "
Sep 11 16:58:54.497801 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.497596    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls\") pod \"d766b2f0-9eec-46e8-a152-653641963131\" (UID: \"d766b2f0-9eec-46e8-a152-653641963131\") "
Sep 11 16:58:54.497854 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.497828    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d766b2f0-9eec-46e8-a152-653641963131" (UID: "d766b2f0-9eec-46e8-a152-653641963131"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:54.499610 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.499581    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d766b2f0-9eec-46e8-a152-653641963131" (UID: "d766b2f0-9eec-46e8-a152-653641963131"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:58:54.499714 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.499678    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs" (OuterVolumeSpecName: "kube-api-access-pmphs") pod "d766b2f0-9eec-46e8-a152-653641963131" (UID: "d766b2f0-9eec-46e8-a152-653641963131"). InnerVolumeSpecName "kube-api-access-pmphs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:54.598686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.598650    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d766b2f0-9eec-46e8-a152-653641963131-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:58:54.598686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.598679    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmphs\" (UniqueName: \"kubernetes.io/projected/d766b2f0-9eec-46e8-a152-653641963131-kube-api-access-pmphs\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:58:54.598686 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:54.598689    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d766b2f0-9eec-46e8-a152-653641963131-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 16:58:55.313233 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.313201    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s" event={"ID":"d766b2f0-9eec-46e8-a152-653641963131","Type":"ContainerDied","Data":"d2668a010dfbd6ebaa78d70e2e6b8d45c1107ce765c569f1dde1f13a5e0d42e1"}
Sep 11 16:58:55.313627 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.313245    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"
Sep 11 16:58:55.313627 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.313265    2566 scope.go:117] "RemoveContainer" containerID="84fc9af0bb85a3fcf6f20473e61a93d953bd37907f48463ec240bb8232cfab27"
Sep 11 16:58:55.322626 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.322607    2566 scope.go:117] "RemoveContainer" containerID="cfadd07d1c98f990461d6a3f9f5a8984750d21fbeee96c0e351b318788b695e1"
Sep 11 16:58:55.334974 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.334940    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:58:55.339603 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:55.339582    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-20f8f-predictor-65795d6ffd-lqj2s"]
Sep 11 16:58:56.111757 ip-10-0-128-17 kubenswrapper[2566]: I0911 16:58:56.111724    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d766b2f0-9eec-46e8-a152-653641963131" path="/var/lib/kubelet/pods/d766b2f0-9eec-46e8-a152-653641963131/volumes"
Sep 11 17:02:18.869364 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:02:18.869252    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 17:02:18.881261 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:02:18.881237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 17:06:21.158603 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:21.158561    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 17:06:21.159215 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:21.158982    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container" containerID="cri-o://72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c" gracePeriod=30
Sep 11 17:06:21.159369 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:21.159315    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kube-rbac-proxy" containerID="cri-o://54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3" gracePeriod=30
Sep 11 17:06:21.395612 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:21.395583    2566 generic.go:356] "Generic (PLEG): container finished" podID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerID="54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3" exitCode=2
Sep 11 17:06:21.395750 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:21.395628    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerDied","Data":"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"}
Sep 11 17:06:23.906563 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.906538    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 17:06:23.984488 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.984423    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhkjz\" (UniqueName: \"kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz\") pod \"839e821c-fa44-488b-ad3a-1754ded0cb23\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") "
Sep 11 17:06:23.984488 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.984454    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls\") pod \"839e821c-fa44-488b-ad3a-1754ded0cb23\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") "
Sep 11 17:06:23.984657 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.984498    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config\") pod \"839e821c-fa44-488b-ad3a-1754ded0cb23\" (UID: \"839e821c-fa44-488b-ad3a-1754ded0cb23\") "
Sep 11 17:06:23.984887 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.984863    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "839e821c-fa44-488b-ad3a-1754ded0cb23" (UID: "839e821c-fa44-488b-ad3a-1754ded0cb23"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:06:23.986450 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.986427    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz" (OuterVolumeSpecName: "kube-api-access-hhkjz") pod "839e821c-fa44-488b-ad3a-1754ded0cb23" (UID: "839e821c-fa44-488b-ad3a-1754ded0cb23"). InnerVolumeSpecName "kube-api-access-hhkjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:23.986529 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:23.986501    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "839e821c-fa44-488b-ad3a-1754ded0cb23" (UID: "839e821c-fa44-488b-ad3a-1754ded0cb23"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:06:24.085092 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.085070    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhkjz\" (UniqueName: \"kubernetes.io/projected/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-api-access-hhkjz\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.085092 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.085088    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/839e821c-fa44-488b-ad3a-1754ded0cb23-proxy-tls\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.085211 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.085099    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/839e821c-fa44-488b-ad3a-1754ded0cb23-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-17.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.410733 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.410698    2566 generic.go:356] "Generic (PLEG): container finished" podID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerID="72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c" exitCode=0
Sep 11 17:06:24.410894 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.410754    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"
Sep 11 17:06:24.410894 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.410784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerDied","Data":"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"}
Sep 11 17:06:24.410894 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.410860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb" event={"ID":"839e821c-fa44-488b-ad3a-1754ded0cb23","Type":"ContainerDied","Data":"0eb113fcec3e819c7571f61b4ea20386932957f9f78e522ba175256c4a2c133b"}
Sep 11 17:06:24.411058 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.410902    2566 scope.go:117] "RemoveContainer" containerID="54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"
Sep 11 17:06:24.419126 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.419109    2566 scope.go:117] "RemoveContainer" containerID="72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"
Sep 11 17:06:24.426980 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.426940    2566 scope.go:117] "RemoveContainer" containerID="54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"
Sep 11 17:06:24.427283 ip-10-0-128-17 kubenswrapper[2566]: E0911 17:06:24.427240    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3\": container with ID starting with 54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3 not found: ID does not exist" containerID="54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"
Sep 11 17:06:24.427283 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.427269    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3"} err="failed to get container status \"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3\": rpc error: code = NotFound desc = could not find container \"54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3\": container with ID starting with 54c0eb52f9b3751027d44ef90f90ba59100644a8183be061786eea6864fce9a3 not found: ID does not exist"
Sep 11 17:06:24.427414 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.427291    2566 scope.go:117] "RemoveContainer" containerID="72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"
Sep 11 17:06:24.427574 ip-10-0-128-17 kubenswrapper[2566]: E0911 17:06:24.427555    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c\": container with ID starting with 72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c not found: ID does not exist" containerID="72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"
Sep 11 17:06:24.427622 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.427604    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c"} err="failed to get container status \"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c\": rpc error: code = NotFound desc = could not find container \"72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c\": container with ID starting with 72366f60d59c230c405065231849bc08e90ab7be951f004dea6d0665b2ab1f5c not found: ID does not exist"
Sep 11 17:06:24.428581 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.428560    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 17:06:24.432258 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:24.432238    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-482a1-predictor-686985f998-wdxbb"]
Sep 11 17:06:26.111579 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:26.111545    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" path="/var/lib/kubelet/pods/839e821c-fa44-488b-ad3a-1754ded0cb23/volumes"
Sep 11 17:06:56.787464 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:56.787344    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcf56_913732e3-f194-491d-9b20-84dcaf859bff/global-pull-secret-syncer/0.log"
Sep 11 17:06:56.900043 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:56.900016    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qmtsv_a7598365-5a86-4eb8-8fb0-18d230ebf3aa/konnectivity-agent/0.log"
Sep 11 17:06:56.956522 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:06:56.956495    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-17.ec2.internal_64a62c92ee6be90122c38a626910285c/haproxy/0.log"
Sep 11 17:07:01.352128 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.352098    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/alertmanager/0.log"
Sep 11 17:07:01.375164 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.375091    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/config-reloader/0.log"
Sep 11 17:07:01.424236 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.424200    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/kube-rbac-proxy-web/0.log"
Sep 11 17:07:01.473109 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.473068    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/kube-rbac-proxy/0.log"
Sep 11 17:07:01.500220 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.500192    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/kube-rbac-proxy-metric/0.log"
Sep 11 17:07:01.519040 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.519007    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/prom-label-proxy/0.log"
Sep 11 17:07:01.539598 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.539578    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da5d1545-9219-479d-9130-98f45e9b6b76/init-config-reloader/0.log"
Sep 11 17:07:01.576379 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.576352    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-shcnn_5b018bb4-0df4-4046-be03-00d7950c316c/cluster-monitoring-operator/0.log"
Sep 11 17:07:01.666094 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.665997    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-794b87fc68-m6shv_2bc4a8d0-a075-46b2-8612-5fb6b27723c3/metrics-server/0.log"
Sep 11 17:07:01.779170 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.779145    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2cj5_1cd40cd2-32ce-453a-bff5-639d8f9c7ed1/node-exporter/0.log"
Sep 11 17:07:01.799301 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.799281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2cj5_1cd40cd2-32ce-453a-bff5-639d8f9c7ed1/kube-rbac-proxy/0.log"
Sep 11 17:07:01.823393 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.823371    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2cj5_1cd40cd2-32ce-453a-bff5-639d8f9c7ed1/init-textfile/0.log"
Sep 11 17:07:01.915649 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.915619    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sgdb2_5663307d-b526-4545-a021-888486bd380d/kube-rbac-proxy-main/0.log"
Sep 11 17:07:01.935443 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.935363    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sgdb2_5663307d-b526-4545-a021-888486bd380d/kube-rbac-proxy-self/0.log"
Sep 11 17:07:01.972843 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:01.972815    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sgdb2_5663307d-b526-4545-a021-888486bd380d/openshift-state-metrics/0.log"
Sep 11 17:07:02.190165 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:02.190086    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-qb5j7_18a30c8f-24dc-4c49-ac72-60a469a43b5c/prometheus-operator-admission-webhook/0.log"
Sep 11 17:07:02.216159 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:02.216128    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f57dd5d9c-bnvbp_ad7f59b0-b129-439c-a3bd-765b7a3023d8/telemeter-client/0.log"
Sep 11 17:07:02.233029 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:02.233011    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f57dd5d9c-bnvbp_ad7f59b0-b129-439c-a3bd-765b7a3023d8/reload/0.log"
Sep 11 17:07:02.261504 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:02.261483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f57dd5d9c-bnvbp_ad7f59b0-b129-439c-a3bd-765b7a3023d8/kube-rbac-proxy/0.log"
Sep 11 17:07:03.640492 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:03.640461    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-s2r4s_500f2611-8683-4f7a-a160-4d156709660b/networking-console-plugin/0.log"
Sep 11 17:07:04.084800 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:04.084771    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 17:07:04.090681 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:04.090660    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/2.log"
Sep 11 17:07:04.965641 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:04.965613    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-7kz4h_899e2f43-ccc0-4a47-ade4-b1f2ce465c30/volume-data-source-validator/0.log"
Sep 11 17:07:05.015134 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015103    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"]
Sep 11 17:07:05.015510 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015495    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015511    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015521    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015526    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015535    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015541    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015547    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015555 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015552    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015561    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015566    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015580    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015585    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015680    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015692    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d766b2f0-9eec-46e8-a152-653641963131" containerName="kserve-container"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015701    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015712    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kserve-container"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015719    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a436a79-c6a0-4ae0-bcc7-d77a3e7f7ee5" containerName="kube-rbac-proxy"
Sep 11 17:07:05.015794 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.015727    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="839e821c-fa44-488b-ad3a-1754ded0cb23" containerName="kserve-container"
Sep 11 17:07:05.019053 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.019034    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.021740 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.021717    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"openshift-service-ca.crt\""
Sep 11 17:07:05.021851 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.021831    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"kube-root-ca.crt\""
Sep 11 17:07:05.022722 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.022707    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8n4t\"/\"default-dockercfg-rjl6t\""
Sep 11 17:07:05.029540 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.029517    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"]
Sep 11 17:07:05.093904 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.093877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-podres\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.094086 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.093909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ns8\" (UniqueName: \"kubernetes.io/projected/d77d7172-ad00-49bc-8606-5db7f634fe49-kube-api-access-c8ns8\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.094086 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.093935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-proc\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.094086 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.094051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-sys\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.094196 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.094124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-lib-modules\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195239 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-lib-modules\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195378 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-podres\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195378 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195276    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ns8\" (UniqueName: \"kubernetes.io/projected/d77d7172-ad00-49bc-8606-5db7f634fe49-kube-api-access-c8ns8\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195378 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-proc\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195378 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-sys\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195518 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-lib-modules\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195518 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-sys\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195518 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-podres\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.195518 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.195413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d77d7172-ad00-49bc-8606-5db7f634fe49-proc\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.203438 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.203413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ns8\" (UniqueName: \"kubernetes.io/projected/d77d7172-ad00-49bc-8606-5db7f634fe49-kube-api-access-c8ns8\") pod \"perf-node-gather-daemonset-4f6xs\" (UID: \"d77d7172-ad00-49bc-8606-5db7f634fe49\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.329943 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.329914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:05.600432 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.600407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"]
Sep 11 17:07:05.602897 ip-10-0-128-17 kubenswrapper[2566]: W0911 17:07:05.602869    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd77d7172_ad00_49bc_8606_5db7f634fe49.slice/crio-4c49847f7b5a26b30b8994db9c5c5d175c6bdf8bc9e5b49d3e021774f751c63d WatchSource:0}: Error finding container 4c49847f7b5a26b30b8994db9c5c5d175c6bdf8bc9e5b49d3e021774f751c63d: Status 404 returned error can't find the container with id 4c49847f7b5a26b30b8994db9c5c5d175c6bdf8bc9e5b49d3e021774f751c63d
Sep 11 17:07:05.812935 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.812904    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-56nbt_15e4ace1-d742-46af-a8f8-515f8df58776/dns/0.log"
Sep 11 17:07:05.829196 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.829162    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-56nbt_15e4ace1-d742-46af-a8f8-515f8df58776/kube-rbac-proxy/0.log"
Sep 11 17:07:05.959770 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:05.959700    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rbxbl_4b554964-07b2-4935-9234-e36967e235d7/dns-node-resolver/0.log"
Sep 11 17:07:06.441380 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:06.441349    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97z2j_9ce9172b-8309-4a61-980b-f98dbcdf1da4/node-ca/0.log"
Sep 11 17:07:06.605894 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:06.605867    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs" event={"ID":"d77d7172-ad00-49bc-8606-5db7f634fe49","Type":"ContainerStarted","Data":"e57efd6240a73fca9c77b0998ff8be7d0c46ca67dee38680bffd3c124b145169"}
Sep 11 17:07:06.606115 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:06.605915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs" event={"ID":"d77d7172-ad00-49bc-8606-5db7f634fe49","Type":"ContainerStarted","Data":"4c49847f7b5a26b30b8994db9c5c5d175c6bdf8bc9e5b49d3e021774f751c63d"}
Sep 11 17:07:06.606115 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:06.605978    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:06.626020 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:06.625981    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs" podStartSLOduration=1.625950204 podStartE2EDuration="1.625950204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:06.623359515 +0000 UTC m=+2989.038842497" watchObservedRunningTime="2026-09-11 17:07:06.625950204 +0000 UTC m=+2989.041433186"
Sep 11 17:07:07.141527 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.141497    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ff58f9bcb-zfvn2_31c36e7b-09da-41bf-bea8-2aba716d5629/router/0.log"
Sep 11 17:07:07.477769 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.477691    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vkpj2_17522b22-a2c6-4428-aeba-537dca1c65f5/serve-healthcheck-canary/0.log"
Sep 11 17:07:07.780696 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.780671    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-lzxh9_b1766afd-bc87-4584-9d5d-4d2a8b77711c/insights-operator/0.log"
Sep 11 17:07:07.781424 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.781407    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-lzxh9_b1766afd-bc87-4584-9d5d-4d2a8b77711c/insights-operator/1.log"
Sep 11 17:07:07.847852 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.847818    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x58vr_29b0a960-b5f5-4c45-8547-bb1e00e0be73/kube-rbac-proxy/0.log"
Sep 11 17:07:07.863687 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.863662    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x58vr_29b0a960-b5f5-4c45-8547-bb1e00e0be73/exporter/0.log"
Sep 11 17:07:07.880520 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:07.880504    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x58vr_29b0a960-b5f5-4c45-8547-bb1e00e0be73/extractor/0.log"
Sep 11 17:07:10.063879 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:10.063849    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-656b75fc8d-svtds_d8008d68-df77-47fb-98e7-405ccfd32ec8/manager/0.log"
Sep 11 17:07:10.090656 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:10.090632    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-6xcgg_54db910f-d070-4637-84b9-e9ae0d8bbe20/manager/0.log"
Sep 11 17:07:10.106328 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:10.106303    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-mrntx_b1960cf1-3a8a-4156-ad63-1c8f0e30c0fd/server/0.log"
Sep 11 17:07:10.183454 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:10.183428    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-lkjvs_f1a54806-fb78-4ac6-8925-9a4fd45c2443/manager/0.log"
Sep 11 17:07:12.620545 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:12.620520    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-4f6xs"
Sep 11 17:07:14.213772 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:14.213701    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-jsz5c_6eac51b9-5627-4d01-b33c-efb8655ab211/kube-storage-version-migrator-operator/1.log"
Sep 11 17:07:14.214550 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:14.214514    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-jsz5c_6eac51b9-5627-4d01-b33c-efb8655ab211/kube-storage-version-migrator-operator/0.log"
Sep 11 17:07:15.394573 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.394543    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/kube-multus-additional-cni-plugins/0.log"
Sep 11 17:07:15.412516 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.412495    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/egress-router-binary-copy/0.log"
Sep 11 17:07:15.428214 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.428167    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/cni-plugins/0.log"
Sep 11 17:07:15.444298 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.444281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/bond-cni-plugin/0.log"
Sep 11 17:07:15.460589 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.460565    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/routeoverride-cni/0.log"
Sep 11 17:07:15.477791 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.477772    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/whereabouts-cni-bincopy/0.log"
Sep 11 17:07:15.496162 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.496143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wznqf_4c64e971-ff67-44a8-959e-48fc454e6d36/whereabouts-cni/0.log"
Sep 11 17:07:15.524531 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.524509    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n78rn_33c04e0f-899f-42a0-8f19-a6725cae80df/kube-multus/0.log"
Sep 11 17:07:15.632447 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.632426    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dmr9d_57db61b1-242f-476a-b2be-d0636b8328b5/network-metrics-daemon/0.log"
Sep 11 17:07:15.650539 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:15.650476    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dmr9d_57db61b1-242f-476a-b2be-d0636b8328b5/kube-rbac-proxy/0.log"
Sep 11 17:07:16.856604 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.856535    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/ovn-controller/0.log"
Sep 11 17:07:16.869209 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.869186    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/ovn-acl-logging/0.log"
Sep 11 17:07:16.887664 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.887641    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/kube-rbac-proxy-node/0.log"
Sep 11 17:07:16.904293 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.904269    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/kube-rbac-proxy-ovn-metrics/0.log"
Sep 11 17:07:16.922085 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.922067    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/northd/0.log"
Sep 11 17:07:16.941955 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.941940    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/nbdb/0.log"
Sep 11 17:07:16.959397 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.959381    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/sbdb/0.log"
Sep 11 17:07:16.996616 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:16.996596    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh8vf_52603598-ccf0-4a97-b81b-101edb0d16b4/ovnkube-controller/0.log"
Sep 11 17:07:17.852705 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:17.852676    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-6f8b768455-2krvv_a91c33b5-2bd1-4486-84a3-0fe1facdf7b3/check-endpoints/0.log"
Sep 11 17:07:17.871084 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:17.871061    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-87g4d_3136b33e-fd4d-4743-98c0-043730b3d595/network-check-target-container/0.log"
Sep 11 17:07:18.719271 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:18.719204    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-pw92x_e137e4d3-027f-4a47-9de3-018983101c0b/iptables-alerter/0.log"
Sep 11 17:07:18.899650 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:18.899560    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 17:07:18.915495 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:18.915477    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gm9tb_f935efa5-8978-4a0e-9860-18cee115cf92/console-operator/1.log"
Sep 11 17:07:19.310844 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:19.310812    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-jk6j8_61003777-8036-4800-b51c-8eb33a3efb4f/tuned/0.log"
Sep 11 17:07:20.995189 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:20.995157    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-2fbhd_719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a/cluster-samples-operator/0.log"
Sep 11 17:07:21.008358 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:21.008329    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-2fbhd_719a71b5-0e60-4bb9-af75-5e4f1c1e1d1a/cluster-samples-operator-watch/0.log"
Sep 11 17:07:21.909465 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:21.909436    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-cf7c776b7-cdlgj_e58035b6-c5c3-434c-8a77-060704f45d0b/service-ca-operator/1.log"
Sep 11 17:07:21.910189 ip-10-0-128-17 kubenswrapper[2566]: I0911 17:07:21.910167    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-cf7c776b7-cdlgj_e58035b6-c5c3-434c-8a77-060704f45d0b/service-ca-operator/0.log"