Sep 17 17:31:58.540510 ip-10-0-128-248 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 17:31:58.540525 ip-10-0-128-248 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 17:31:58.540535 ip-10-0-128-248 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 17:31:58.540746 ip-10-0-128-248 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 17:32:08.574785 ip-10-0-128-248 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 17:32:08.574804 ip-10-0-128-248 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a06064bc7a3d48878d217dd0dc518ffb -- Sep 17 17:34:10.651437 ip-10-0-128-248 systemd[1]: Starting Kubernetes Kubelet... Sep 17 17:34:11.002832 ip-10-0-128-248 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:34:11.002832 ip-10-0-128-248 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 17:34:11.002832 ip-10-0-128-248 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:34:11.002832 ip-10-0-128-248 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 17:34:11.002832 ip-10-0-128-248 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:34:11.003935 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.003844 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 17:34:11.009091 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009074 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.009091 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009092 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009097 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009101 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009104 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009124 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009127 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009130 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009133 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009135 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009138 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009141 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009143 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009152 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009156 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009160 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009162 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009165 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009167 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009170 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009173 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.009181 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009176 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009178 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009181 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009186 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009190 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009194 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009197 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009200 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009203 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009206 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009209 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009212 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009215 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009218 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009222 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009226 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009229 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009232 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009236 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.009654 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009239 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009241 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009244 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009246 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009249 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009251 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009254 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009256 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009259 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009261 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009264 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009266 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009269 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009273 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009276 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009280 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009283 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009286 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009289 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.010232 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009291 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009294 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009296 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009299 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009302 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009304 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009307 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009309 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009312 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009314 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009317 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009319 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009322 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009325 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009327 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009330 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009332 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009335 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009337 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009341 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.010726 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009343 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009346 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009349 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009352 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009354 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009357 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009360 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009362 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009365 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009368 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009371 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009374 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009376 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009379 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009381 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009384 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009386 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009389 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009391 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.009394 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.011225 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010098 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010104 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010107 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010110 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010127 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010130 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010133 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010136 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010138 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010141 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010144 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010147 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010150 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010154 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010158 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010162 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010165 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010168 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010171 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010174 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.011743 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010179 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010182 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010185 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010188 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010191 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010193 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010196 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010199 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010202 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010204 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010207 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010211 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010215 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010218 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010221 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010224 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010227 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010231 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010235 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010238 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.012267 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010241 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010244 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010247 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010250 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010253 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010256 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010259 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010263 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010266 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010269 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010271 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010274 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010277 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010281 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010284 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010287 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010290 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010293 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010295 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.012791 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010298 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010301 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010303 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010307 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010309 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010312 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010314 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010317 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010320 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010322 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010325 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010327 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010330 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010333 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010336 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010338 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010341 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010344 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010346 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010349 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.013283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010352 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010354 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010357 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010360 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010362 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010365 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010368 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010370 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010373 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010376 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010378 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010381 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010383 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010386 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010389 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010391 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010394 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010396 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010399 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010401 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.013773 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010784 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010789 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010792 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010795 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010798 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010801 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010804 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010806 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010809 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010812 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010814 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010817 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010820 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010822 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010825 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010828 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010831 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010834 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010836 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.014275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010839 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010842 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010844 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010848 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010850 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010853 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010855 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010858 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010861 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010864 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010866 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010869 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010871 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010874 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010877 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010879 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010882 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010884 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010889 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010892 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.014740 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010895 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010898 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010902 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010906 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010909 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010912 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010915 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010918 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010921 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010925 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010931 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010934 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010937 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010940 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010943 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010945 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010948 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010951 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010953 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.015241 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010956 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010959 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010961 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010963 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010966 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010969 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010971 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010974 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010976 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010979 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010982 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.010997 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011002 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011006 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011009 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011012 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011015 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011017 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011022 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011024 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.015696 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011027 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011030 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011033 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011036 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011039 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011042 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011044 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011047 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011050 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011052 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011055 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011058 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011060 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011063 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011065 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011068 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011070 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011073 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011075 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011078 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.016213 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011081 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011166 2583 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011200 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011207 2583 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011212 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011217 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011221 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011226 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011231 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011234 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011238 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011242 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011245 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011248 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011251 2583 flags.go:64] FLAG: --cgroup-root="" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011255 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011258 2583 flags.go:64] FLAG: --client-ca-file="" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011261 2583 flags.go:64] FLAG: --cloud-provider="external" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011264 2583 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011269 2583 flags.go:64] FLAG: --cluster-domain="" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011272 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011275 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011278 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011282 2583 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 17:34:11.016704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011285 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011288 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011291 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011294 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011297 2583 flags.go:64] FLAG: --contention-profiling="false" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011300 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011303 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011306 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011310 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011313 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011317 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011320 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011323 2583 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011326 2583 flags.go:64] FLAG: --enable-server="true" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011329 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011334 2583 flags.go:64] FLAG: --event-burst="100" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011337 2583 flags.go:64] FLAG: --event-qps="50" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011340 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011343 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011346 2583 flags.go:64] FLAG: --eviction-hard="" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011350 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011353 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011356 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011359 2583 flags.go:64] FLAG: --eviction-soft="" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011362 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 17:34:11.017305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011365 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011368 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011371 2583 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011374 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011377 2583 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011380 2583 flags.go:64] FLAG: --feature-gates="" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011384 2583 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011387 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011390 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011393 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011396 2583 flags.go:64] FLAG: --healthz-port="10248" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011399 2583 flags.go:64] FLAG: --help="false" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011402 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011405 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011408 2583 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011411 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011414 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011418 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011420 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011423 2583 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011426 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011430 2583 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011433 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011436 2583 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 17:34:11.017915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011439 2583 flags.go:64] FLAG: --kube-reserved="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011442 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011445 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011448 2583 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011450 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011453 2583 flags.go:64] FLAG: --lock-file="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011456 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011459 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011462 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011467 2583 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011470 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011473 2583 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011476 2583 flags.go:64] FLAG: --logging-format="text" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011479 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011482 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011485 2583 flags.go:64] FLAG: --manifest-url="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011488 2583 flags.go:64] FLAG: --manifest-url-header="" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011492 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011495 2583 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011499 2583 flags.go:64] FLAG: --max-pods="110" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011502 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011505 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011508 2583 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011511 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011514 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 17:34:11.018533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011517 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011520 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011526 2583 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011530 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011533 2583 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011536 2583 flags.go:64] FLAG: --pod-cidr="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011539 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011545 2583 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011548 2583 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011550 2583 flags.go:64] FLAG: --pods-per-core="0" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011553 2583 flags.go:64] FLAG: --port="10250" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011556 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011559 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03048b8afdc775702" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011562 2583 flags.go:64] FLAG: --qos-reserved="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011565 2583 flags.go:64] FLAG: --read-only-port="10255" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011568 2583 flags.go:64] FLAG: --register-node="true" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011571 2583 flags.go:64] FLAG: --register-with-taints="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011574 2583 flags.go:64] FLAG: --registry-burst="10" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011577 2583 flags.go:64] FLAG: --registry-qps="5" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011580 2583 flags.go:64] FLAG: --reserved-cpus="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011583 2583 flags.go:64] FLAG: --reserved-memory="" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011586 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011589 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011592 2583 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011595 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 17:34:11.019179 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011598 2583 flags.go:64] FLAG: --runonce="false" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011601 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011604 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011607 2583 flags.go:64] FLAG: --seccomp-default="false" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011609 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011612 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011615 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011618 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011621 2583 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011623 2583 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011626 2583 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011632 2583 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011635 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011638 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011642 2583 flags.go:64] FLAG: --system-cgroups="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011645 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011650 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011653 2583 flags.go:64] FLAG: --tls-cert-file="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011656 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011660 2583 flags.go:64] FLAG: --tls-min-version="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011663 2583 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011666 2583 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011669 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011671 2583 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011674 2583 flags.go:64] FLAG: --v="2" Sep 17 17:34:11.019784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011678 2583 flags.go:64] FLAG: --version="false" Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011682 2583 flags.go:64] FLAG: --vmodule="" Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011686 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.011690 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011804 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011808 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011811 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011814 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011817 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011821 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011823 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011826 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011829 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011831 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011834 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011837 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011840 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011845 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011853 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011858 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.020416 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011862 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011864 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011869 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011872 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011875 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011877 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011880 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011883 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011886 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011888 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011891 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011894 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011897 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011899 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011902 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011905 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011907 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011910 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011912 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011915 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.020922 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011917 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011920 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011923 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011925 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011928 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011930 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011933 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011936 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011938 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011941 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011947 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011949 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011952 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011955 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011957 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011960 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011963 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011965 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011968 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011971 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.021435 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011974 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011976 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011979 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011982 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011984 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011987 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011989 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011992 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011994 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.011997 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012000 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012004 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012007 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012010 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012013 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012016 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012019 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012021 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012025 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.022275 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012028 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012030 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012033 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012039 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012042 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012045 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012047 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012050 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012053 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012062 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012065 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012068 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012070 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012073 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012076 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012079 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012081 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012084 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012086 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.022958 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012089 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012092 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012094 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012097 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.012100 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.012939 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.019925 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.019948 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.019968 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.019975 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020032 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020036 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020040 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020043 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020046 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.023492 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020049 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020052 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020054 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020057 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020060 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020062 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020065 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020068 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020071 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020074 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020077 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020080 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020083 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020087 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020091 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020094 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020098 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020101 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020104 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.023875 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020107 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020123 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020126 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020129 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020132 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020136 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020141 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020145 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020148 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020151 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020154 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020156 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020159 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020161 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020164 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020167 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020169 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020172 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020175 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.024461 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020177 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020180 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020182 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020185 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020188 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020190 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020193 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020196 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020199 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020202 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020205 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020208 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020210 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020213 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020217 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020220 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020223 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020226 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020229 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.024968 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020233 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020235 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020238 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020241 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020244 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020247 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020249 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020252 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020254 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020257 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020259 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020262 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020265 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020267 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020270 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020272 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020275 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020277 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020280 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020283 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.025474 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020286 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020288 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020291 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020293 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020296 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020299 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020301 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020304 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020306 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020309 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020312 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020315 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020318 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020321 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020324 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020326 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.026332 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020329 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.020334 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020449 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020454 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020458 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020462 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020465 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020468 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020471 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020474 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020476 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020479 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020482 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020485 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020488 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:34:11.026885 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020490 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020493 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020496 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020499 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020502 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020504 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020507 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020509 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020512 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020514 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020517 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020520 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020523 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020525 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020528 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020531 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020533 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020537 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020540 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:34:11.027282 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020543 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020545 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020548 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020551 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020553 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020556 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020558 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020561 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020564 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020566 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020569 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020572 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020574 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020576 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020579 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020582 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020585 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020587 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020590 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020592 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:34:11.027893 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020595 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020598 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020600 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020604 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020607 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020610 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020613 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020616 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020618 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020621 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020624 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020626 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020629 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020632 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020634 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020637 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020639 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020642 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020644 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020647 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:34:11.028449 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020650 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020652 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020655 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020657 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020660 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020663 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020665 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020668 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020671 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020673 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020676 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020678 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020681 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020683 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020686 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020688 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020690 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020693 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020696 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:34:11.029030 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020698 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020701 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020704 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020707 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020709 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020712 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020714 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:11.020717 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.020722 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.020846 2583 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.024540 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.025375 2583 server.go:1019] "Starting client certificate rotation" Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.025494 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 17:34:11.029582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.025546 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 17:34:11.029930 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.027734 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 17:34:11.029930 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.028892 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 17:34:11.039899 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.039880 2583 log.go:25] "Validated CRI v1 runtime API" Sep 17 17:34:11.041773 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.041753 2583 log.go:25] "Validated CRI v1 image API" Sep 17 17:34:11.042683 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.042670 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 17:34:11.044804 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.044787 2583 fs.go:135] Filesystem UUIDs: map[27d36c76-e5d2-4857-9b09-a544319b6eac:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7aec891d-1902-40a3-92a4-778cd843bbc5:/dev/nvme0n1p4] Sep 17 17:34:11.044849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.044805 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 17:34:11.050467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.050339 2583 manager.go:216] Machine: {Timestamp:2026-09-17 17:34:11.048473329 +0000 UTC m=+0.294514125 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104426 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2092680aa4d7fb6ee865051ecb9b6c SystemUUID:ec209268-0aa4-d7fb-6ee8-65051ecb9b6c BootID:a06064bc-7a3d-4887-8d21-7dd0dc518ffb Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e0:6d:c9:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e0:6d:c9:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:10:3e:96:fb:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 17:34:11.050467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.050462 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 17:34:11.050583 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.050552 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 17:34:11.053387 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.053354 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 17:34:11.053601 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.053390 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 17:34:11.054051 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.054040 2583 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 17:34:11.054086 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.054053 2583 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 17:34:11.054086 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.054061 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 17:34:11.054636 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.054625 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 17:34:11.054717 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.054709 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 17:34:11.055610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.055590 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 17:34:11.057605 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057583 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 17 17:34:11.057723 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057714 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 17:34:11.057763 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057751 2583 kubelet.go:486] "Attempting to sync node with API server" Sep 17 17:34:11.057792 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057769 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 17:34:11.057792 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057784 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 17:34:11.057852 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057795 2583 kubelet.go:398] "Adding apiserver pod source" Sep 17 17:34:11.057852 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.057803 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 17:34:11.058285 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.058274 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 17:34:11.058328 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.058291 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 17:34:11.061802 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.061761 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 17 17:34:11.064200 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.064185 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 17:34:11.064266 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.064216 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 17:34:11.065288 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065277 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065292 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065298 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065303 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065312 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065317 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065331 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065336 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 17:34:11.065342 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065343 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 17:34:11.065540 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065348 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 17:34:11.065540 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065357 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 17:34:11.065540 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065362 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 17:34:11.065923 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065913 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 17:34:11.065923 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.065921 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 17:34:11.069602 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.069587 2583 server.go:1268] "Started kubelet" Sep 17 17:34:11.069701 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.069678 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 17:34:11.069755 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.069668 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 17:34:11.069755 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.069739 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 17:34:11.070259 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.070238 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 17:34:11.070358 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.070259 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 17:34:11.070442 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.070408 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 17:34:11.070481 ip-10-0-128-248 systemd[1]: Started Kubernetes Kubelet. Sep 17 17:34:11.070958 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.070939 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 17:34:11.071848 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.071835 2583 server.go:310] "Adding debug handlers to kubelet server" Sep 17 17:34:11.073708 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.073690 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 17:34:11.076844 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.076825 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 17:34:11.077314 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.077299 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 17:34:11.077415 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.077397 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.077892 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.077875 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 17:34:11.077985 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.077872 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 17:34:11.077985 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.077909 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 17:34:11.078069 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078016 2583 reconstruct.go:97] "Volume reconstruction finished" Sep 17 17:34:11.078069 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078026 2583 reconciler.go:29] "Reconciler: start to sync state" Sep 17 17:34:11.078069 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078062 2583 factory.go:55] Registering systemd factory Sep 17 17:34:11.078241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078073 2583 factory.go:223] Registration of the systemd container factory successfully Sep 17 17:34:11.078348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078333 2583 factory.go:153] Registering CRI-O factory Sep 17 17:34:11.078392 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078352 2583 factory.go:223] Registration of the crio container factory successfully Sep 17 17:34:11.078434 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078426 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 17:34:11.078491 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078445 2583 factory.go:103] Registering Raw factory Sep 17 17:34:11.078491 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078467 2583 manager.go:1196] Started watching for new ooms in manager Sep 17 17:34:11.078888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.078876 2583 manager.go:349] Starting recovery of all containers Sep 17 17:34:11.079127 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.078303 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-248.ec2.internal.18d62c27ba6f69a1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-248.ec2.internal,UID:ip-10-0-128-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-248.ec2.internal,},FirstTimestamp:2026-09-17 17:34:11.069561249 +0000 UTC m=+0.315602046,LastTimestamp:2026-09-17 17:34:11.069561249 +0000 UTC m=+0.315602046,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-248.ec2.internal,}" Sep 17 17:34:11.083145 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.083101 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 17:34:11.083252 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.083189 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 17:34:11.089759 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.089735 2583 manager.go:354] Recovery completed Sep 17 17:34:11.091761 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.091740 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 17 17:34:11.094686 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.094674 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.096322 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096303 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.096402 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096340 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.096402 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096356 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.096977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096959 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 17:34:11.096977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096974 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 17:34:11.096977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096976 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.097102 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.096990 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 17 17:34:11.097684 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.097669 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.097722 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.097698 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.097722 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.097709 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.098192 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.098181 2583 policy_none.go:49] "None policy: Start" Sep 17 17:34:11.098237 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.098196 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 17:34:11.098237 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.098205 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 17:34:11.098749 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.098668 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-248.ec2.internal.18d62c27bc07c51b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-248.ec2.internal,UID:ip-10-0-128-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-248.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-248.ec2.internal,},FirstTimestamp:2026-09-17 17:34:11.096323355 +0000 UTC m=+0.342364154,LastTimestamp:2026-09-17 17:34:11.096323355 +0000 UTC m=+0.342364154,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-248.ec2.internal,}" Sep 17 17:34:11.099224 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.099211 2583 policy_none.go:47] "Start" Sep 17 17:34:11.107463 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.107369 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-248.ec2.internal.18d62c27bc082309 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-248.ec2.internal,UID:ip-10-0-128-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-248.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-248.ec2.internal,},FirstTimestamp:2026-09-17 17:34:11.096347401 +0000 UTC m=+0.342388199,LastTimestamp:2026-09-17 17:34:11.096347401 +0000 UTC m=+0.342388199,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-248.ec2.internal,}" Sep 17 17:34:11.109859 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.109795 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-248.ec2.internal.18d62c27bc085b47 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-248.ec2.internal,UID:ip-10-0-128-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-248.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-248.ec2.internal,},FirstTimestamp:2026-09-17 17:34:11.096361799 +0000 UTC m=+0.342402596,LastTimestamp:2026-09-17 17:34:11.096361799 +0000 UTC m=+0.342402596,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-248.ec2.internal,}" Sep 17 17:34:11.110429 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.110406 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2gr5f" Sep 17 17:34:11.119177 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.119158 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2gr5f" Sep 17 17:34:11.121659 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.120745 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-248.ec2.internal.18d62c27bc07c51b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-248.ec2.internal.18d62c27bc07c51b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-248.ec2.internal,UID:ip-10-0-128-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-248.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-248.ec2.internal,},FirstTimestamp:2026-09-17 17:34:11.096323355 +0000 UTC m=+0.342364154,LastTimestamp:2026-09-17 17:34:11.09768423 +0000 UTC m=+0.343725032,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-248.ec2.internal,}" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.140870 2583 manager.go:337] "Starting Device Plugin manager" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.140925 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.140940 2583 server.go:87] "Starting device plugin registration server" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.141197 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.141211 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.141298 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.141440 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.141451 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.141959 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 17:34:11.149821 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.142003 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.216054 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.216017 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 17:34:11.217212 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.217193 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 17:34:11.217279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.217263 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 17:34:11.217320 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.217293 2583 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 17:34:11.217370 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.217341 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 17:34:11.219922 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.219901 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:34:11.241875 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.241837 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.242794 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.242778 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.242868 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.242808 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.242868 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.242818 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.242868 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.242843 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.255783 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.255734 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.255783 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.255760 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-248.ec2.internal\": node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.270077 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.270047 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.318161 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.318101 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal"] Sep 17 17:34:11.318329 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.318213 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.319634 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.319617 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.319723 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.319647 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.319723 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.319657 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.321753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.321740 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.321874 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.321859 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.321911 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.321890 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.322537 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322520 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.322637 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322545 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.322637 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322559 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.322637 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322605 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.322637 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322624 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.322811 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.322640 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.324533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.324516 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.324604 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.324542 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:11.325290 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.325271 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:11.325373 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.325303 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:11.325373 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.325314 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:11.359970 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.359934 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-248.ec2.internal\" not found" node="ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.364422 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.364402 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-248.ec2.internal\" not found" node="ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.370234 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.370217 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.379547 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.379523 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.379629 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.379566 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.379629 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.379588 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df14a0960708fff328f38437ff677d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-248.ec2.internal\" (UID: \"6df14a0960708fff328f38437ff677d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.470565 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.470534 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.479896 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479873 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.479967 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.479967 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df14a0960708fff328f38437ff677d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-248.ec2.internal\" (UID: \"6df14a0960708fff328f38437ff677d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.480048 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df14a0960708fff328f38437ff677d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-248.ec2.internal\" (UID: \"6df14a0960708fff328f38437ff677d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.480048 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479980 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.480048 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.479980 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b3f0eec76aa0374d16752ca332607f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal\" (UID: \"3b3f0eec76aa0374d16752ca332607f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.571374 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.571344 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.663909 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.663871 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.669027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:11.668887 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:11.672162 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.671989 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.772361 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.772318 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.872926 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.872847 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:11.973382 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:11.973328 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:12.025837 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.025803 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 17:34:12.026288 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.025974 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 17:34:12.073588 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:12.073544 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:12.123584 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.123512 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 17:29:11 +0000 UTC" deadline="2028-04-16 14:01:43.30587633 +0000 UTC" Sep 17 17:34:12.123584 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.123552 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13844h27m31.182329132s" Sep 17 17:34:12.173856 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:12.173823 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:12.179073 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.179049 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:34:12.180380 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.180359 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:34:12.180467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.180394 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:34:12.180467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.180404 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:34:12.273939 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:12.273899 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df14a0960708fff328f38437ff677d3.slice/crio-da2795cb917a3b104dbbd3d2574e3fa24ce9e9114f5b6e75b197769be8797247 WatchSource:0}: Error finding container da2795cb917a3b104dbbd3d2574e3fa24ce9e9114f5b6e75b197769be8797247: Status 404 returned error can't find the container with id da2795cb917a3b104dbbd3d2574e3fa24ce9e9114f5b6e75b197769be8797247 Sep 17 17:34:12.274100 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:12.273951 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-248.ec2.internal\" not found" Sep 17 17:34:12.274648 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:12.274622 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3f0eec76aa0374d16752ca332607f7.slice/crio-08f92544644708b75eb016a81b26b59043849d3d7208a09b6ce45bd1f91ccff1 WatchSource:0}: Error finding container 08f92544644708b75eb016a81b26b59043849d3d7208a09b6ce45bd1f91ccff1: Status 404 returned error can't find the container with id 08f92544644708b75eb016a81b26b59043849d3d7208a09b6ce45bd1f91ccff1 Sep 17 17:34:12.280220 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.280202 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 17:34:12.309707 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.309672 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:34:12.357078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.357045 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:34:12.377686 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.377624 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" Sep 17 17:34:12.407092 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.407063 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 17:34:12.408026 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.408012 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" Sep 17 17:34:12.427037 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.427012 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 17:34:12.576398 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.576362 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:34:12.966873 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:12.966842 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:34:13.058890 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.058846 2583 apiserver.go:52] "Watching apiserver" Sep 17 17:34:13.071951 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.071743 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 17:34:13.072226 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.072199 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-fbxm5","openshift-dns/node-resolver-s9gfm","openshift-image-registry/node-ca-97t8m","openshift-multus/multus-additional-cni-plugins-44pd9","kube-system/global-pull-secret-syncer-kw8q4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal","openshift-multus/multus-ds6zl","openshift-multus/network-metrics-daemon-nkskg","openshift-network-diagnostics/network-check-target-lxqq9","openshift-network-operator/iptables-alerter-426rl","openshift-ovn-kubernetes/ovnkube-node-jmrg9","kube-system/konnectivity-agent-wwnvl","kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92"] Sep 17 17:34:13.074774 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.074742 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.077222 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.077200 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.078289 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.078269 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 17:34:13.079424 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.079401 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.081745 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.081724 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.083823 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.083802 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 17:34:13.084368 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.084192 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.084368 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.084287 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:13.084590 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.084572 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mcvcq\"" Sep 17 17:34:13.085495 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.085470 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 17:34:13.085588 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.085560 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.085894 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.085874 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rvvsn\"" Sep 17 17:34:13.085966 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.085930 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 17:34:13.086286 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.085884 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.086286 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.086064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 17:34:13.086286 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.086152 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.086915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.086894 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.088628 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088609 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-multus\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088738 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088646 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-hostroot\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088738 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-log-socket\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088738 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjkjc\" (UniqueName: \"kubernetes.io/projected/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-kube-api-access-gjkjc\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.088738 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cni-binary-copy\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088742 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-multus-certs\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-slash\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088785 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-node-log\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088811 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088836 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2429979-1441-417e-8241-16e35406a3a4-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088861 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088881 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-etc-kubernetes\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088903 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-netns\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.088936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-serviceca\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.088977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-k8s-cni-cncf-io\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089017 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-daemon-config\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-script-lib\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-system-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089190 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cnibin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-socket-dir-parent\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089269 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-kubelet\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089294 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-conf-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089315 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-kubelet\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089342 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089364 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-systemd-units\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-systemd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-bin\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089462 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-netd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.089509 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-config\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-env-overrides\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8dc\" (UniqueName: \"kubernetes.io/projected/d2429979-1441-417e-8241-16e35406a3a4-kube-api-access-dl8dc\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-host\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb9qf\" (UniqueName: \"kubernetes.io/projected/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-kube-api-access-jb9qf\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-etc-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089662 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-var-lib-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089685 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-ovn\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-netns\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.090269 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.089766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-bin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.090705 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.090491 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:13.090705 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.090553 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:13.090969 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.090899 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.091060 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.090975 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 17:34:13.091060 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091046 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z7kbm\"" Sep 17 17:34:13.091197 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091073 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.091197 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.090902 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 17:34:13.091305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091209 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.091305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091233 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 17:34:13.091407 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.090909 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6c2fl\"" Sep 17 17:34:13.091407 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091349 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 17:34:13.091846 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091716 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 17:34:13.091846 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091730 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.091846 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091779 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 17:34:13.091846 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.091782 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.092400 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.092382 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zt5hz\"" Sep 17 17:34:13.093226 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.092894 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.093226 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.092955 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:13.096195 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.096173 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.098423 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.098394 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.103104 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.100407 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.103104 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.100542 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.103104 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.101040 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-szphp\"" Sep 17 17:34:13.103104 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.102316 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.105686 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.105668 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.106683 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.106643 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.107361 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107301 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.107521 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107493 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:13.107593 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107547 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:13.107884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107683 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gnnqt\"" Sep 17 17:34:13.107884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107695 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zq7ph\"" Sep 17 17:34:13.107884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.107776 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 17:34:13.108645 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.108626 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9k88f\"" Sep 17 17:34:13.111488 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.111197 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 17:34:13.111488 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.111206 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 17:34:13.113517 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.113493 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 17:34:13.133765 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.133429 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-psxsd" Sep 17 17:34:13.142107 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.142076 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-psxsd" Sep 17 17:34:13.179294 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.179267 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 17:34:13.190884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190858 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2429979-1441-417e-8241-16e35406a3a4-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.190884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znxsf\" (UniqueName: \"kubernetes.io/projected/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-kube-api-access-znxsf\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190928 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysconfig\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-host\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-k8s-cni-cncf-io\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.190981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-tmp\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-system-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191039 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cnibin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191059 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-socket-dir-parent\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191071 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-k8s-cni-cncf-io\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-kubelet\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191137 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-system-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191142 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cnibin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191142 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-kubelet\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-env-overrides\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191228 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-socket-dir-parent\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191266 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-socket-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191312 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191322 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-netd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dc\" (UniqueName: \"kubernetes.io/projected/d2429979-1441-417e-8241-16e35406a3a4-kube-api-access-dl8dc\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-host\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191400 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191405 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-netd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-sys-fs\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-kubelet-config\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191456 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-host\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191474 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-etc-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191496 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a041459a-331e-4af8-a415-9f072005e2ec-host-slash\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.191656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191518 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-run\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191565 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7s76\" (UniqueName: \"kubernetes.io/projected/a041459a-331e-4af8-a415-9f072005e2ec-kube-api-access-r7s76\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191566 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-env-overrides\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191598 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191612 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-etc-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vv68\" (UniqueName: \"kubernetes.io/projected/ab89a7e9-37cb-46e9-8535-e1d6173de675-kube-api-access-5vv68\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191659 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcknw\" (UniqueName: \"kubernetes.io/projected/3889437a-f56a-4156-b469-3f4d5ac0afd6-kube-api-access-xcknw\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-netns\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-bin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191763 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-netns\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjkjc\" (UniqueName: \"kubernetes.io/projected/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-kube-api-access-gjkjc\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a041459a-331e-4af8-a415-9f072005e2ec-iptables-alerter-script\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191825 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95c6bf95-de5b-41a0-b59f-79e4af0016c0-konnectivity-ca\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191833 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-bin\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191852 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cni-binary-copy\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.192356 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-node-log\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zhzc\" (UniqueName: \"kubernetes.io/projected/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-kube-api-access-6zhzc\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191988 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-node-log\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.191988 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192005 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192032 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-dbus\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-etc-kubernetes\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-netns\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-serviceca\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192208 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-etc-kubernetes\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-netns\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95c6bf95-de5b-41a0-b59f-79e4af0016c0-agent-certs\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-lib-modules\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.193156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-daemon-config\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-script-lib\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192332 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-cni-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192382 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5s7d\" (UniqueName: \"kubernetes.io/projected/610a5ea2-1a65-4740-8366-66869cc3ba6b-kube-api-access-p5s7d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-conf-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-kubelet\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192487 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-binary-copy\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192496 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-cni-binary-copy\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-tmp-dir\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192525 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-modprobe-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-conf\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192554 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192557 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-serviceca\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192583 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-systemd-units\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-kubelet\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.193888 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-systemd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192607 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-conf-dir\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-systemd\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-bin\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-config\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192675 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-cni-bin\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192692 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-systemd-units\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-system-cni-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192730 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-cnibin\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-registration-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-multus-daemon-config\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-sys\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb9qf\" (UniqueName: \"kubernetes.io/projected/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-kube-api-access-jb9qf\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.192951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-hosts-file\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-systemd\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-tuned\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-var-lib-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.194671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-ovn\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193176 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-config\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193186 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-var-lib-openvswitch\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-kubernetes\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193208 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2429979-1441-417e-8241-16e35406a3a4-ovnkube-script-lib\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193230 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-run-ovn\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193265 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-var-lib-kubelet\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-multus\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-hostroot\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193340 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-log-socket\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193379 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-var-lib-cni-multus\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-device-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193427 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-hostroot\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193444 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193474 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-log-socket\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-multus-certs\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.195486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-slash\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.196231 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193518 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-host-run-multus-certs\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.196231 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.193577 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2429979-1441-417e-8241-16e35406a3a4-host-slash\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.196661 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.196639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2429979-1441-417e-8241-16e35406a3a4-ovn-node-metrics-cert\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.200035 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.199751 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjkjc\" (UniqueName: \"kubernetes.io/projected/1f9ccc9b-6d34-4923-8320-ad4586f8f1d0-kube-api-access-gjkjc\") pod \"node-ca-97t8m\" (UID: \"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0\") " pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.200035 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.199967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8dc\" (UniqueName: \"kubernetes.io/projected/d2429979-1441-417e-8241-16e35406a3a4-kube-api-access-dl8dc\") pod \"ovnkube-node-jmrg9\" (UID: \"d2429979-1441-417e-8241-16e35406a3a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.200802 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.200782 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb9qf\" (UniqueName: \"kubernetes.io/projected/c3851eaf-b94a-4016-a49b-b45a5efb0c0f-kube-api-access-jb9qf\") pod \"multus-ds6zl\" (UID: \"c3851eaf-b94a-4016-a49b-b45a5efb0c0f\") " pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.222234 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.222131 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" event={"ID":"6df14a0960708fff328f38437ff677d3","Type":"ContainerStarted","Data":"da2795cb917a3b104dbbd3d2574e3fa24ce9e9114f5b6e75b197769be8797247"} Sep 17 17:34:13.223329 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.223305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" event={"ID":"3b3f0eec76aa0374d16752ca332607f7","Type":"ContainerStarted","Data":"08f92544644708b75eb016a81b26b59043849d3d7208a09b6ce45bd1f91ccff1"} Sep 17 17:34:13.293815 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.293780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znxsf\" (UniqueName: \"kubernetes.io/projected/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-kube-api-access-znxsf\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.293996 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.293824 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysconfig\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.293996 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.293896 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysconfig\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294130 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-host\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294130 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-tmp\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294130 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-socket-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294139 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-host\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294222 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-sys-fs\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-kubelet-config\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294252 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294270 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294269 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a041459a-331e-4af8-a415-9f072005e2ec-host-slash\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294279 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-socket-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-run\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294312 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-sys-fs\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294327 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7s76\" (UniqueName: \"kubernetes.io/projected/a041459a-331e-4af8-a415-9f072005e2ec-kube-api-access-r7s76\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.294335 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vv68\" (UniqueName: \"kubernetes.io/projected/ab89a7e9-37cb-46e9-8535-e1d6173de675-kube-api-access-5vv68\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294362 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a041459a-331e-4af8-a415-9f072005e2ec-host-slash\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294362 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-kubelet-config\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcknw\" (UniqueName: \"kubernetes.io/projected/3889437a-f56a-4156-b469-3f4d5ac0afd6-kube-api-access-xcknw\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294333 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-run\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.294432 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:13.794386741 +0000 UTC m=+3.040427527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a041459a-331e-4af8-a415-9f072005e2ec-iptables-alerter-script\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294563 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95c6bf95-de5b-41a0-b59f-79e4af0016c0-konnectivity-ca\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zhzc\" (UniqueName: \"kubernetes.io/projected/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-kube-api-access-6zhzc\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.294655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-dbus\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294709 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95c6bf95-de5b-41a0-b59f-79e4af0016c0-agent-certs\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-lib-modules\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5s7d\" (UniqueName: \"kubernetes.io/projected/610a5ea2-1a65-4740-8366-66869cc3ba6b-kube-api-access-p5s7d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-binary-copy\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-tmp-dir\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-modprobe-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-conf\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-system-cni-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-cnibin\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.294988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-registration-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-sys\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-hosts-file\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.295427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295055 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-systemd\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-tuned\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295108 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-kubernetes\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a041459a-331e-4af8-a415-9f072005e2ec-iptables-alerter-script\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295155 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-var-lib-kubelet\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-device-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295220 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-sys\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295403 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-tmp-dir\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-kubernetes\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295056 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95c6bf95-de5b-41a0-b59f-79e4af0016c0-konnectivity-ca\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295503 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295626 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-lib-modules\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295658 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-var-lib-kubelet\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295717 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.296228 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295791 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295835 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26893c92-4fe2-4746-b1e0-43972e90d6de-dbus\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295846 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-device-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295894 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.295920 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-system-cni-dir\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.295969 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:13.795952657 +0000 UTC m=+3.041993445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295996 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab89a7e9-37cb-46e9-8535-e1d6173de675-cni-binary-copy\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296023 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-modprobe-d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3889437a-f56a-4156-b469-3f4d5ac0afd6-registration-dir\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.295505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-systemd\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296062 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab89a7e9-37cb-46e9-8535-e1d6173de675-cnibin\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296080 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-sysctl-conf\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-hosts-file\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.297014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.296529 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-tmp\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.298425 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.298404 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/610a5ea2-1a65-4740-8366-66869cc3ba6b-etc-tuned\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.298596 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.298572 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95c6bf95-de5b-41a0-b59f-79e4af0016c0-agent-certs\") pod \"konnectivity-agent-wwnvl\" (UID: \"95c6bf95-de5b-41a0-b59f-79e4af0016c0\") " pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.302103 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.302040 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:13.302103 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.302067 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:13.302103 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.302083 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:13.302338 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.302166 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:13.802148627 +0000 UTC m=+3.048189428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:13.304314 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.304290 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znxsf\" (UniqueName: \"kubernetes.io/projected/2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0-kube-api-access-znxsf\") pod \"node-resolver-s9gfm\" (UID: \"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0\") " pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.304473 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.304391 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcknw\" (UniqueName: \"kubernetes.io/projected/3889437a-f56a-4156-b469-3f4d5ac0afd6-kube-api-access-xcknw\") pod \"aws-ebs-csi-driver-node-btq92\" (UID: \"3889437a-f56a-4156-b469-3f4d5ac0afd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.304473 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.304434 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zhzc\" (UniqueName: \"kubernetes.io/projected/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-kube-api-access-6zhzc\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.305442 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.305399 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7s76\" (UniqueName: \"kubernetes.io/projected/a041459a-331e-4af8-a415-9f072005e2ec-kube-api-access-r7s76\") pod \"iptables-alerter-426rl\" (UID: \"a041459a-331e-4af8-a415-9f072005e2ec\") " pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.305846 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.305816 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vv68\" (UniqueName: \"kubernetes.io/projected/ab89a7e9-37cb-46e9-8535-e1d6173de675-kube-api-access-5vv68\") pod \"multus-additional-cni-plugins-44pd9\" (UID: \"ab89a7e9-37cb-46e9-8535-e1d6173de675\") " pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.306417 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.306396 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5s7d\" (UniqueName: \"kubernetes.io/projected/610a5ea2-1a65-4740-8366-66869cc3ba6b-kube-api-access-p5s7d\") pod \"tuned-fbxm5\" (UID: \"610a5ea2-1a65-4740-8366-66869cc3ba6b\") " pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.388555 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.388515 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ds6zl" Sep 17 17:34:13.395549 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.395518 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:13.406367 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.406338 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97t8m" Sep 17 17:34:13.411237 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.411210 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44pd9" Sep 17 17:34:13.420026 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.420005 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" Sep 17 17:34:13.427763 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.427741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s9gfm" Sep 17 17:34:13.436939 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.436915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-426rl" Sep 17 17:34:13.444557 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.444525 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" Sep 17 17:34:13.450218 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.450197 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:13.797987 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.797953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:13.798197 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.798009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:13.798197 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.798098 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:13.798197 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.798179 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.798165365 +0000 UTC m=+4.044206161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:13.798369 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.798099 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:13.798369 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.798243 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.798231779 +0000 UTC m=+4.044272566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:13.898828 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:13.898787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:13.899001 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.898979 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:13.899081 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.899006 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:13.899081 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.899019 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:13.899209 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:13.899092 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.899071197 +0000 UTC m=+4.145111998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:14.100927 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.100891 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2429979_1441_417e_8241_16e35406a3a4.slice/crio-ec82416dcfe1acdfbe1a1ed689b795844242f7173dda2a7729d1f9bd39807d2b WatchSource:0}: Error finding container ec82416dcfe1acdfbe1a1ed689b795844242f7173dda2a7729d1f9bd39807d2b: Status 404 returned error can't find the container with id ec82416dcfe1acdfbe1a1ed689b795844242f7173dda2a7729d1f9bd39807d2b Sep 17 17:34:14.103912 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.103207 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3889437a_f56a_4156_b469_3f4d5ac0afd6.slice/crio-83550d6dde78d59066aa9490cb7015324c408df14ad4b9612bed78ba0ad0a3da WatchSource:0}: Error finding container 83550d6dde78d59066aa9490cb7015324c408df14ad4b9612bed78ba0ad0a3da: Status 404 returned error can't find the container with id 83550d6dde78d59066aa9490cb7015324c408df14ad4b9612bed78ba0ad0a3da Sep 17 17:34:14.126060 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.126026 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab89a7e9_37cb_46e9_8535_e1d6173de675.slice/crio-ae20da43268036b2649ced6c8cc4936a62c52047daac6266d18bce79d9a83936 WatchSource:0}: Error finding container ae20da43268036b2649ced6c8cc4936a62c52047daac6266d18bce79d9a83936: Status 404 returned error can't find the container with id ae20da43268036b2649ced6c8cc4936a62c52047daac6266d18bce79d9a83936 Sep 17 17:34:14.126611 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.126594 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda041459a_331e_4af8_a415_9f072005e2ec.slice/crio-3eb54a45bf9e529fcae5ab47bcd961e3fa64971b98d7bf022665a2746f7dc641 WatchSource:0}: Error finding container 3eb54a45bf9e529fcae5ab47bcd961e3fa64971b98d7bf022665a2746f7dc641: Status 404 returned error can't find the container with id 3eb54a45bf9e529fcae5ab47bcd961e3fa64971b98d7bf022665a2746f7dc641 Sep 17 17:34:14.128061 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.128026 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610a5ea2_1a65_4740_8366_66869cc3ba6b.slice/crio-0021da17e651091ca1527447a19ab55def2a1fa25c051911fd1ce59258943857 WatchSource:0}: Error finding container 0021da17e651091ca1527447a19ab55def2a1fa25c051911fd1ce59258943857: Status 404 returned error can't find the container with id 0021da17e651091ca1527447a19ab55def2a1fa25c051911fd1ce59258943857 Sep 17 17:34:14.129042 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.129020 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9ccc9b_6d34_4923_8320_ad4586f8f1d0.slice/crio-4200bc4796a91fa16ee3e2d631de2ddca690f96b479cfa434ebb45a047420c9b WatchSource:0}: Error finding container 4200bc4796a91fa16ee3e2d631de2ddca690f96b479cfa434ebb45a047420c9b: Status 404 returned error can't find the container with id 4200bc4796a91fa16ee3e2d631de2ddca690f96b479cfa434ebb45a047420c9b Sep 17 17:34:14.129677 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.129620 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1a7c3c_7d1b_41c2_bc7f_28634f1234a0.slice/crio-ec3b16087cf80547a49891ab78ee13e7de122f8a01ed01709719161862162f27 WatchSource:0}: Error finding container ec3b16087cf80547a49891ab78ee13e7de122f8a01ed01709719161862162f27: Status 404 returned error can't find the container with id ec3b16087cf80547a49891ab78ee13e7de122f8a01ed01709719161862162f27 Sep 17 17:34:14.130980 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.130914 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c6bf95_de5b_41a0_b59f_79e4af0016c0.slice/crio-8b9a4b38df5847bd312011312c3a28a063d6e226b40e2a804cef0acf4111e268 WatchSource:0}: Error finding container 8b9a4b38df5847bd312011312c3a28a063d6e226b40e2a804cef0acf4111e268: Status 404 returned error can't find the container with id 8b9a4b38df5847bd312011312c3a28a063d6e226b40e2a804cef0acf4111e268 Sep 17 17:34:14.133228 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:14.133208 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3851eaf_b94a_4016_a49b_b45a5efb0c0f.slice/crio-5111f254f5db76593eb467d0c94246b649710cc3e2433d37384f25c053209d09 WatchSource:0}: Error finding container 5111f254f5db76593eb467d0c94246b649710cc3e2433d37384f25c053209d09: Status 404 returned error can't find the container with id 5111f254f5db76593eb467d0c94246b649710cc3e2433d37384f25c053209d09 Sep 17 17:34:14.145218 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.145184 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:13 +0000 UTC" deadline="2028-05-20 03:09:03.793010501 +0000 UTC" Sep 17 17:34:14.145315 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.145219 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14649h34m49.647796514s" Sep 17 17:34:14.218396 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.218367 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:14.218533 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.218492 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:14.226903 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.226870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerStarted","Data":"ae20da43268036b2649ced6c8cc4936a62c52047daac6266d18bce79d9a83936"} Sep 17 17:34:14.228338 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.228314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"ec82416dcfe1acdfbe1a1ed689b795844242f7173dda2a7729d1f9bd39807d2b"} Sep 17 17:34:14.231753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.231729 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" event={"ID":"6df14a0960708fff328f38437ff677d3","Type":"ContainerStarted","Data":"d9532d3e55bf0d333ac48cd3d900a969be0aa68058e6b9cc14121c6a326a2149"} Sep 17 17:34:14.234719 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.234663 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wwnvl" event={"ID":"95c6bf95-de5b-41a0-b59f-79e4af0016c0","Type":"ContainerStarted","Data":"8b9a4b38df5847bd312011312c3a28a063d6e226b40e2a804cef0acf4111e268"} Sep 17 17:34:14.245150 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.245101 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97t8m" event={"ID":"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0","Type":"ContainerStarted","Data":"4200bc4796a91fa16ee3e2d631de2ddca690f96b479cfa434ebb45a047420c9b"} Sep 17 17:34:14.247970 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.247915 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-248.ec2.internal" podStartSLOduration=2.2478985160000002 podStartE2EDuration="2.247898516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:14.247273871 +0000 UTC m=+3.493314675" watchObservedRunningTime="2026-09-17 17:34:14.247898516 +0000 UTC m=+3.493939322" Sep 17 17:34:14.248598 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.248570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" event={"ID":"610a5ea2-1a65-4740-8366-66869cc3ba6b","Type":"ContainerStarted","Data":"0021da17e651091ca1527447a19ab55def2a1fa25c051911fd1ce59258943857"} Sep 17 17:34:14.251567 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.251524 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-426rl" event={"ID":"a041459a-331e-4af8-a415-9f072005e2ec","Type":"ContainerStarted","Data":"3eb54a45bf9e529fcae5ab47bcd961e3fa64971b98d7bf022665a2746f7dc641"} Sep 17 17:34:14.253240 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.253219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" event={"ID":"3889437a-f56a-4156-b469-3f4d5ac0afd6","Type":"ContainerStarted","Data":"83550d6dde78d59066aa9490cb7015324c408df14ad4b9612bed78ba0ad0a3da"} Sep 17 17:34:14.256688 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.256666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ds6zl" event={"ID":"c3851eaf-b94a-4016-a49b-b45a5efb0c0f","Type":"ContainerStarted","Data":"5111f254f5db76593eb467d0c94246b649710cc3e2433d37384f25c053209d09"} Sep 17 17:34:14.257652 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.257630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s9gfm" event={"ID":"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0","Type":"ContainerStarted","Data":"ec3b16087cf80547a49891ab78ee13e7de122f8a01ed01709719161862162f27"} Sep 17 17:34:14.810384 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.810344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:14.810561 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.810407 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:14.810623 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.810560 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:14.810674 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.810625 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:16.810606189 +0000 UTC m=+6.056646995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:14.811023 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.811006 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:14.811098 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.811057 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:16.811042535 +0000 UTC m=+6.057083322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:14.911142 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:14.910954 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:14.911142 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.911147 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:14.911354 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.911171 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:14.911354 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.911184 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:14.911354 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:14.911274 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:16.911230147 +0000 UTC m=+6.157270938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:15.146313 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.146167 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:13 +0000 UTC" deadline="2028-03-12 06:25:02.98052196 +0000 UTC" Sep 17 17:34:15.146313 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.146228 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12996h50m47.834297433s" Sep 17 17:34:15.220551 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.220515 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:15.220731 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:15.220655 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:15.221078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.221049 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:15.221200 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:15.221177 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:15.276831 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.276794 2583 generic.go:356] "Generic (PLEG): container finished" podID="3b3f0eec76aa0374d16752ca332607f7" containerID="0766f3c9075d6a511d96520e27c25feff1682a207ea080a76fd88872e69f549c" exitCode=0 Sep 17 17:34:15.277014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:15.276880 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" event={"ID":"3b3f0eec76aa0374d16752ca332607f7","Type":"ContainerDied","Data":"0766f3c9075d6a511d96520e27c25feff1682a207ea080a76fd88872e69f549c"} Sep 17 17:34:16.218162 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:16.218107 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:16.218634 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.218250 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:16.299708 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:16.299672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" event={"ID":"3b3f0eec76aa0374d16752ca332607f7","Type":"ContainerStarted","Data":"b34b50123fa57034b45e0fcfe04ab1fc381f4954cd3b5aac24617e221ba86939"} Sep 17 17:34:16.830467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:16.830398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:16.830467 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:16.830463 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:16.830716 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.830633 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:16.830716 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.830696 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:20.83067861 +0000 UTC m=+10.076719399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:16.831238 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.831095 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:16.831238 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.831166 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:20.831148901 +0000 UTC m=+10.077189691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:16.931828 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:16.931792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:16.932058 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.931950 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:16.932058 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.931970 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:16.932058 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.931983 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:16.932058 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:16.932051 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:20.93203158 +0000 UTC m=+10.178072414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:17.218773 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:17.218688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:17.219244 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:17.218827 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:17.219244 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:17.218857 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:17.219244 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:17.218959 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:18.217721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:18.217682 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:18.217903 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:18.217819 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:19.218085 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:19.217751 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:19.218085 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:19.217762 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:19.218085 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:19.217927 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:19.218085 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:19.217987 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:20.218664 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:20.218476 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:20.218664 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.218603 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:20.867024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:20.866985 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:20.867236 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:20.867069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:20.867236 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.867202 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:20.867236 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.867212 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:20.867495 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.867277 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:28.867257039 +0000 UTC m=+18.113297841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:20.867495 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.867296 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:28.86728733 +0000 UTC m=+18.113328119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:20.967975 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:20.967930 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:20.968163 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.968095 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:20.968163 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.968135 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:20.968163 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.968149 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:20.968329 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:20.968215 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:28.96819561 +0000 UTC m=+18.214236397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:21.219456 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:21.219375 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:21.219884 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:21.219491 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:21.219884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:21.219850 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:21.220018 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:21.219950 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:22.218741 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:22.218243 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:22.218741 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:22.218378 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:23.218195 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:23.218158 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:23.218195 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:23.218182 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:23.218710 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:23.218305 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:23.218710 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:23.218405 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:24.217691 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:24.217658 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:24.217858 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:24.217766 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:25.218177 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:25.218130 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:25.218584 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:25.218144 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:25.218584 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:25.218314 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:25.218584 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:25.218413 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:26.217475 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:26.217441 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:26.217662 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:26.217564 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:27.218285 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:27.218244 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:27.218761 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:27.218387 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:27.218761 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:27.218437 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:27.218935 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:27.218910 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:28.217532 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:28.217482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:28.217721 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:28.217610 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:28.930882 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:28.930835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:28.931365 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:28.930896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:28.931365 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:28.930986 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:28.931365 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:28.930996 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:28.931365 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:28.931066 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:34:44.931046297 +0000 UTC m=+34.177087081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:28.931365 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:28.931143 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:34:44.93110917 +0000 UTC m=+34.177149972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:29.031456 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:29.031417 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:29.031632 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.031603 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:29.031632 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.031626 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:29.031759 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.031636 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:29.031759 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.031693 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:45.031679171 +0000 UTC m=+34.277719958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:29.218424 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:29.218336 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:29.218581 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:29.218340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:29.218581 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.218481 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:29.218581 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:29.218563 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:30.218301 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:30.218260 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:30.218729 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:30.218403 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:31.218557 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:31.218518 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:31.219081 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:31.218622 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:31.219081 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:31.218686 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:31.219081 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:31.218817 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:32.218064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.217886 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:32.218246 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:32.218176 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:32.334651 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.334557 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" event={"ID":"3889437a-f56a-4156-b469-3f4d5ac0afd6","Type":"ContainerStarted","Data":"7f1ac55d41f54d9703aeff8f67b3d5cba3bee65f4546a111f093f44b75ba40d6"} Sep 17 17:34:32.336309 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.336283 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ds6zl" event={"ID":"c3851eaf-b94a-4016-a49b-b45a5efb0c0f","Type":"ContainerStarted","Data":"9cc7de526091c9417831b641f7c6b3640c1458637918a8e3d949e2a043c554ca"} Sep 17 17:34:32.338313 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.338286 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s9gfm" event={"ID":"2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0","Type":"ContainerStarted","Data":"3c142a70e40f6367df4b6336d58456f95365325ad396dd19d551a1550b49244a"} Sep 17 17:34:32.339945 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.339917 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="66d1e8b626a9f13e4c884e445bc37f4758b73b9a701703a13a979142396ae88f" exitCode=0 Sep 17 17:34:32.340025 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.340001 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"66d1e8b626a9f13e4c884e445bc37f4758b73b9a701703a13a979142396ae88f"} Sep 17 17:34:32.342406 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.342382 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"0ac581a6eeba3142f785b79b5b295638af6d226a5c58d0ad59c12613f79acfb0"} Sep 17 17:34:32.342487 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.342449 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"a715ce7d12f2c8edb4305248fe62500c7351bc2fcc6a01e49e89edb52ff1d8d4"} Sep 17 17:34:32.344005 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.343985 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wwnvl" event={"ID":"95c6bf95-de5b-41a0-b59f-79e4af0016c0","Type":"ContainerStarted","Data":"b0b013df4688d58ced4baa5908a51e6eb14d00df9851f33c52340c3cde48942c"} Sep 17 17:34:32.345551 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.345522 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97t8m" event={"ID":"1f9ccc9b-6d34-4923-8320-ad4586f8f1d0","Type":"ContainerStarted","Data":"726d51bd51f80d18a5ff20fd35529b39614a4d21ec05dcca02b0a3e816ccf42b"} Sep 17 17:34:32.346952 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.346931 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" event={"ID":"610a5ea2-1a65-4740-8366-66869cc3ba6b","Type":"ContainerStarted","Data":"a33f9ca7286280b8d68ea0d1d855c6028e68eebce63f2175367b5d4e561ee740"} Sep 17 17:34:32.351884 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.351844 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-248.ec2.internal" podStartSLOduration=20.351829115 podStartE2EDuration="20.351829115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:16.316055148 +0000 UTC m=+5.562095954" watchObservedRunningTime="2026-09-17 17:34:32.351829115 +0000 UTC m=+21.597869919" Sep 17 17:34:32.351991 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.351928 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ds6zl" podStartSLOduration=3.760791997 podStartE2EDuration="21.351918809s" totalImagesPullingTime="17.591126812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:32.351599281 +0000 UTC m=+21.597640098" watchObservedRunningTime="2026-09-17 17:34:32.351918809 +0000 UTC m=+21.597959615" Sep 17 17:34:32.368066 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.368020 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wwnvl" podStartSLOduration=3.859500934 podStartE2EDuration="21.368006064s" totalImagesPullingTime="17.50850513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:32.367560547 +0000 UTC m=+21.613601352" watchObservedRunningTime="2026-09-17 17:34:32.368006064 +0000 UTC m=+21.614046869" Sep 17 17:34:32.381939 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.381896 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-97t8m" podStartSLOduration=3.791997474 podStartE2EDuration="21.381881489s" totalImagesPullingTime="17.589884015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:32.381410174 +0000 UTC m=+21.627450993" watchObservedRunningTime="2026-09-17 17:34:32.381881489 +0000 UTC m=+21.627922345" Sep 17 17:34:32.394217 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.394171 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s9gfm" podStartSLOduration=3.805248981 podStartE2EDuration="21.394155761s" totalImagesPullingTime="17.58890678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:32.394147632 +0000 UTC m=+21.640188430" watchObservedRunningTime="2026-09-17 17:34:32.394155761 +0000 UTC m=+21.640196563" Sep 17 17:34:32.411849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:32.411802 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fbxm5" podStartSLOduration=3.8209060900000003 podStartE2EDuration="21.411787811s" totalImagesPullingTime="17.590881721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:32.411430515 +0000 UTC m=+21.657471337" watchObservedRunningTime="2026-09-17 17:34:32.411787811 +0000 UTC m=+21.657828616" Sep 17 17:34:33.218837 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.218611 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:33.218988 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.218650 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:33.218988 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:33.218957 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:33.219101 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:33.219042 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:33.355024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.354857 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"c6fcee4a5c863007053dad1cad0a98d74086a835c0208f8cb2dad740cdd168d7"} Sep 17 17:34:33.355024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.354937 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"3255bd3a13d6d7edb302c09f7ba5812e8bbb3c44e945702a89073c415339892d"} Sep 17 17:34:33.355024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.354973 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"cef6afa1935874ade2040f3be1958f6e784e335b1e80c0a4ada1121ebc20ec6c"} Sep 17 17:34:33.355024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.355007 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"fe74fef66276910f7ccc2f5e631fb86f0cfac75dcb1f0bb2e6bac87d1e2486dd"} Sep 17 17:34:33.413503 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:33.413324 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 17:34:34.152305 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.152171 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T17:34:33.413345487Z","UUID":"ecad66f7-2f05-4741-8a41-5c0d4b85c037","Handler":null,"Name":"","Endpoint":""} Sep 17 17:34:34.154184 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.154053 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 17:34:34.154184 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.154087 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 17:34:34.218156 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.218105 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:34.218337 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:34.218259 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:34.287041 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.287000 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:34.287663 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.287642 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:34.359886 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.359853 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-426rl" event={"ID":"a041459a-331e-4af8-a415-9f072005e2ec","Type":"ContainerStarted","Data":"e7a7896d3e078a719effbf81520ba93615e26152f46f52ac2d86a7ea576c3643"} Sep 17 17:34:34.362649 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.362620 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" event={"ID":"3889437a-f56a-4156-b469-3f4d5ac0afd6","Type":"ContainerStarted","Data":"e08383ca273e4884ecb14f0c445443d11bc92b6319644892db8aca8abb917631"} Sep 17 17:34:34.362849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.362832 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:34.363442 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.363387 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wwnvl" Sep 17 17:34:34.400745 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:34.400683 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-426rl" podStartSLOduration=5.805429315 podStartE2EDuration="23.400659906s" totalImagesPullingTime="17.595230591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:34.381521356 +0000 UTC m=+23.627562164" watchObservedRunningTime="2026-09-17 17:34:34.400659906 +0000 UTC m=+23.646700715" Sep 17 17:34:35.217863 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:35.217832 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:35.218014 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:35.217832 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:35.218014 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:35.217950 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:35.218104 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:35.218063 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:35.368975 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:35.368941 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"d02e140fb2069dd38ebca5b489bb1d3398b00040aa2796c4ba22d1d72e799feb"} Sep 17 17:34:35.371198 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:35.371172 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" event={"ID":"3889437a-f56a-4156-b469-3f4d5ac0afd6","Type":"ContainerStarted","Data":"e5f25217e17f641d4442ffea75e7335d4e5fb613f1099a4fc4dedf7ce8773aad"} Sep 17 17:34:35.395169 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:35.395108 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq92" podStartSLOduration=4.777938799 podStartE2EDuration="24.39509092s" totalImagesPullingTime="19.617152121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:35.394613049 +0000 UTC m=+24.640653867" watchObservedRunningTime="2026-09-17 17:34:35.39509092 +0000 UTC m=+24.641131724" Sep 17 17:34:36.218129 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:36.218081 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:36.218307 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:36.218221 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:37.218659 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.218482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:37.219144 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.218570 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:37.219144 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:37.218746 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:37.219144 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:37.218869 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:37.378500 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.378468 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="2545dc4197e14ac375accd2bee214a02d7de2b9792f5df1998299603d00a8683" exitCode=0 Sep 17 17:34:37.378679 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.378552 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"2545dc4197e14ac375accd2bee214a02d7de2b9792f5df1998299603d00a8683"} Sep 17 17:34:37.382510 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.382482 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" event={"ID":"d2429979-1441-417e-8241-16e35406a3a4","Type":"ContainerStarted","Data":"07eb2ef0f8b3ef113b522ed97f2d30aa5cb193a9d647e5425a1c7e235c488501"} Sep 17 17:34:37.382840 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.382816 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:37.398686 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.398614 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:37.436314 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:37.436261 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" podStartSLOduration=8.444758614 podStartE2EDuration="26.436245365s" totalImagesPullingTime="17.991486751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:37.435813401 +0000 UTC m=+26.681854207" watchObservedRunningTime="2026-09-17 17:34:37.436245365 +0000 UTC m=+26.682286169" Sep 17 17:34:38.218139 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.218089 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:38.218372 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:38.218237 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:38.387568 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.387532 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="c6df25d8716cfeb5f3a45d60e6c1d1d52d92e45d486f4641bb593e2dc1107a52" exitCode=0 Sep 17 17:34:38.388281 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.387610 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"c6df25d8716cfeb5f3a45d60e6c1d1d52d92e45d486f4641bb593e2dc1107a52"} Sep 17 17:34:38.388281 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.388083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:38.388281 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.388174 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:38.402629 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.402602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:34:38.892107 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.892076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kw8q4"] Sep 17 17:34:38.892280 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.892202 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:38.892316 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:38.892279 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.932449 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nkskg"] Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.933271 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.933411 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lxqq9"] Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:38.933524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:38.933538 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:38.934214 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:38.933636 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:40.218285 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:40.218032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:40.218660 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:40.218032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:40.218660 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:40.218328 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:40.218660 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:40.218386 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:40.394420 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:40.394387 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="1c1480b03dce2b82e57258830235fb9ad8c76a6fb11adcca03ec8767c70b7072" exitCode=0 Sep 17 17:34:40.394620 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:40.394468 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"1c1480b03dce2b82e57258830235fb9ad8c76a6fb11adcca03ec8767c70b7072"} Sep 17 17:34:41.219392 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:41.219357 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:41.219788 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:41.219493 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:42.217906 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:42.217858 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:42.218094 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:42.218002 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:42.218094 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:42.218060 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:42.218236 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:42.218193 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:43.218452 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:43.218414 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:43.219003 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:43.218558 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:44.218149 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:44.218096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:44.218350 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:44.218096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:44.218350 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.218252 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kw8q4" podUID="26893c92-4fe2-4746-b1e0-43972e90d6de" Sep 17 17:34:44.218350 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.218333 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lxqq9" podUID="51140b5c-ad9c-40c4-8e15-572191352895" Sep 17 17:34:44.945087 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:44.945054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:44.945401 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:44.945149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:44.945401 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.945196 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:44.945401 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.945266 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:44.945401 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.945278 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs podName:cdd47ebc-76f8-4ff4-92a6-39f2700c35be nodeName:}" failed. No retries permitted until 2026-09-17 17:35:16.945256866 +0000 UTC m=+66.191297667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs") pod "network-metrics-daemon-nkskg" (UID: "cdd47ebc-76f8-4ff4-92a6-39f2700c35be") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:34:44.945401 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:44.945305 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret podName:26893c92-4fe2-4746-b1e0-43972e90d6de nodeName:}" failed. No retries permitted until 2026-09-17 17:35:16.945293961 +0000 UTC m=+66.191334763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret") pod "global-pull-secret-syncer-kw8q4" (UID: "26893c92-4fe2-4746-b1e0-43972e90d6de") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:34:45.045656 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.045628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:45.045850 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.045828 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:34:45.045892 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.045863 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:34:45.045892 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.045877 2583 projected.go:196] Error preparing data for projected volume kube-api-access-pt5dj for pod openshift-network-diagnostics/network-check-target-lxqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:45.045963 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.045942 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj podName:51140b5c-ad9c-40c4-8e15-572191352895 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.045921756 +0000 UTC m=+66.291962556 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt5dj" (UniqueName: "kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj") pod "network-check-target-lxqq9" (UID: "51140b5c-ad9c-40c4-8e15-572191352895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:34:45.218308 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.218222 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:45.218453 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.218340 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkskg" podUID="cdd47ebc-76f8-4ff4-92a6-39f2700c35be" Sep 17 17:34:45.412380 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.412191 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="92dd9fd619ecb9814ca7221670c267834b889728421874f78a267f9d05524570" exitCode=0 Sep 17 17:34:45.412559 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.412266 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"92dd9fd619ecb9814ca7221670c267834b889728421874f78a267f9d05524570"} Sep 17 17:34:45.545829 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.545804 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-248.ec2.internal" event="NodeReady" Sep 17 17:34:45.545998 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.545964 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 17:34:45.587105 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.587079 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:34:45.596743 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.596716 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.599549 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.599498 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 17:34:45.599694 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.599587 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqbcm\"" Sep 17 17:34:45.599694 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.599587 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 17:34:45.599694 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.599635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 17:34:45.605709 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.605688 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj"] Sep 17 17:34:45.606675 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.606644 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 17:34:45.610937 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.610920 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8"] Sep 17 17:34:45.611062 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.611045 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" Sep 17 17:34:45.614694 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.614677 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.617052 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.617028 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24"] Sep 17 17:34:45.617193 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.617174 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.620411 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620395 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8n6sw\"" Sep 17 17:34:45.620507 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620412 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.620774 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.620875 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620858 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g5gjf\"" Sep 17 17:34:45.620933 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620909 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 17:34:45.621071 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.620965 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.622605 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.622584 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 17:34:45.623000 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.622980 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc"] Sep 17 17:34:45.623165 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.623132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.627967 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.627946 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.628078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.627993 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hd9k5\"" Sep 17 17:34:45.628078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.627949 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 17:34:45.629033 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.628593 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.634742 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.630565 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb"] Sep 17 17:34:45.634742 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.632383 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.634947 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.634925 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 17:34:45.635078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.635040 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.635227 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.635208 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kjls5\"" Sep 17 17:34:45.637229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.637059 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 17 17:34:45.637766 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.637403 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.638064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.638045 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 17 17:34:45.639131 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.639095 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:34:45.639235 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.639156 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-spbxm"] Sep 17 17:34:45.639297 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.639280 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.646926 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.645653 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 17 17:34:45.646926 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.646254 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.650471 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.650434 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p22z4\"" Sep 17 17:34:45.650649 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.650500 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.651996 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.651972 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r"] Sep 17 17:34:45.654057 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.653149 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.656924 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.656213 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.656924 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.656421 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.656924 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.656622 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 17:34:45.656924 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.656809 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ccfks\"" Sep 17 17:34:45.657190 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.657029 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 17:34:45.657406 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.657384 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-4qtmt"] Sep 17 17:34:45.657695 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.657674 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.662141 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.661918 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 17:34:45.663021 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.662994 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.663215 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.663130 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 17:34:45.663324 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.663309 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.663458 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.663440 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9trxn\"" Sep 17 17:34:45.663527 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.663482 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 17:34:45.667367 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.667345 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-f58f99974-897ql"] Sep 17 17:34:45.667706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.667689 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.670438 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.670416 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.670572 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.670447 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 17:34:45.670572 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.670462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 17:34:45.670684 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.670672 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.672135 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.672093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9x258\"" Sep 17 17:34:45.675797 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.675775 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf"] Sep 17 17:34:45.675900 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.675885 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.676462 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.676445 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 17:34:45.678464 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678446 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 17 17:34:45.678571 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678477 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.678571 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678477 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.678571 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678530 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zlr65\"" Sep 17 17:34:45.678757 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678745 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 17 17:34:45.678850 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678835 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 17 17:34:45.678907 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.678896 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 17 17:34:45.681677 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.681657 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t"] Sep 17 17:34:45.681763 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.681741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" Sep 17 17:34:45.684027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.683989 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.684795 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.684619 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.685414 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.685395 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-n7pmb\"" Sep 17 17:34:45.693872 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.693852 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bsbv6"] Sep 17 17:34:45.693990 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.693969 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.696425 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.696397 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 17:34:45.696425 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.696400 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 17:34:45.696712 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.696697 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-x5zgp\"" Sep 17 17:34:45.707891 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.707866 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7"] Sep 17 17:34:45.708000 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.707980 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.710123 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.710091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 17:34:45.710225 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.710103 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdbm5\"" Sep 17 17:34:45.710225 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.710205 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 17:34:45.719844 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.719823 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg"] Sep 17 17:34:45.719955 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.719939 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.722531 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.722506 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 17 17:34:45.734229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.734207 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qjmqs"] Sep 17 17:34:45.734349 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.734331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.737534 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.737509 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 17 17:34:45.738059 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.738043 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 17 17:34:45.738168 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.738157 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 17 17:34:45.738304 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.738285 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 17 17:34:45.746222 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746200 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746240 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746262 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746276 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746289 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746305 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-spbxm"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746304 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746321 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r"] Sep 17 17:34:45.746337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746336 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bsbv6"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746359 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f58f99974-897ql"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746383 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qjmqs"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746402 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746419 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-4qtmt"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746435 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746450 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t"] Sep 17 17:34:45.746721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.746467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7"] Sep 17 17:34:45.749578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.749561 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 17:34:45.749710 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.749600 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 17:34:45.749710 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.749612 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7dqqw\"" Sep 17 17:34:45.749710 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.749600 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 17:34:45.750640 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.750727 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jp84\" (UniqueName: \"kubernetes.io/projected/46a12da8-7e00-40c0-a736-3071a72c4147-kube-api-access-4jp84\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.750727 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750695 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjql\" (UniqueName: \"kubernetes.io/projected/f753c88b-c4b9-4666-80b2-48a09074e5e8-kube-api-access-fhjql\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.750727 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmvx9\" (UniqueName: \"kubernetes.io/projected/158e5a53-5c15-4402-8a11-7963c8246cfe-kube-api-access-cmvx9\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.750920 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.750920 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f5f068-c587-41ea-b827-a1736453bb8d-serving-cert\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.750920 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e1c9ab-363a-4900-ab06-01740f969943-serving-cert\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.750920 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750858 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xmnn\" (UniqueName: \"kubernetes.io/projected/e0677dda-7088-4f85-91d3-7d07b270b611-kube-api-access-9xmnn\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.750920 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750885 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750929 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq4hm\" (UniqueName: \"kubernetes.io/projected/4207130c-c9e5-4358-9860-7f2ebffa6615-kube-api-access-sq4hm\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.750997 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f753c88b-c4b9-4666-80b2-48a09074e5e8-serving-cert\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpzw\" (UniqueName: \"kubernetes.io/projected/939b0f33-4382-405b-ab91-17a162e03fbb-kube-api-access-mbpzw\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-default-certificate\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrsdw\" (UniqueName: \"kubernetes.io/projected/b914f895-ad9f-4472-88e3-2de946cf4a9f-kube-api-access-qrsdw\") pod \"network-check-source-7cbc45fddc-2kvhf\" (UID: \"b914f895-ad9f-4472-88e3-2de946cf4a9f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751182 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0677dda-7088-4f85-91d3-7d07b270b611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.751247 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-trusted-ca\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85j98\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751455 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjzvd\" (UniqueName: \"kubernetes.io/projected/d7e1c9ab-363a-4900-ab06-01740f969943-kube-api-access-vjzvd\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a12da8-7e00-40c0-a736-3071a72c4147-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751634 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a12da8-7e00-40c0-a736-3071a72c4147-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-snapshots\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751683 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.751706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751760 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9ctp\" (UniqueName: \"kubernetes.io/projected/47f5f068-c587-41ea-b827-a1736453bb8d-kube-api-access-t9ctp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/939b0f33-4382-405b-ab91-17a162e03fbb-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmtw8\" (UniqueName: \"kubernetes.io/projected/f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d-kube-api-access-zmtw8\") pod \"volume-data-source-validator-5c4db8bdf9-bxjrj\" (UID: \"f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-tmp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751891 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-config\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7e1c9ab-363a-4900-ab06-01740f969943-config\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.752229 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.751945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-stats-auth\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.852997 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.852957 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjzvd\" (UniqueName: \"kubernetes.io/projected/d7e1c9ab-363a-4900-ab06-01740f969943-kube-api-access-vjzvd\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.852997 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.852996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a12da8-7e00-40c0-a736-3071a72c4147-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a12da8-7e00-40c0-a736-3071a72c4147-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853043 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-snapshots\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d162d8fa-063a-4f39-8d0c-c665350f1ec0-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853082 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv9kc\" (UniqueName: \"kubernetes.io/projected/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-kube-api-access-zv9kc\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853150 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853176 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9ctp\" (UniqueName: \"kubernetes.io/projected/47f5f068-c587-41ea-b827-a1736453bb8d-kube-api-access-t9ctp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.853279 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.853277 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.353252457 +0000 UTC m=+35.599293258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:34:45.853578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td5jw\" (UniqueName: \"kubernetes.io/projected/e681489d-a82b-44a3-bade-59fe825f1260-kube-api-access-td5jw\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.853578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853438 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/939b0f33-4382-405b-ab91-17a162e03fbb-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.853578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmtw8\" (UniqueName: \"kubernetes.io/projected/f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d-kube-api-access-zmtw8\") pod \"volume-data-source-validator-5c4db8bdf9-bxjrj\" (UID: \"f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" Sep 17 17:34:45.853578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-tmp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.853578 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-config\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e681489d-a82b-44a3-bade-59fe825f1260-cert\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853613 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853615 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7e1c9ab-363a-4900-ab06-01740f969943-config\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-tmp\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853671 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a12da8-7e00-40c0-a736-3071a72c4147-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853702 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-stats-auth\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.853834 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853781 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-snapshots\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47f5f068-c587-41ea-b827-a1736453bb8d-tmp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.853979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b60746a-01ad-4609-a3ef-e31bbdf74407-tmp-dir\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854168 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9b52\" (UniqueName: \"kubernetes.io/projected/2b60746a-01ad-4609-a3ef-e31bbdf74407-kube-api-access-g9b52\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854198 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/939b0f33-4382-405b-ab91-17a162e03fbb-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.854241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jp84\" (UniqueName: \"kubernetes.io/projected/46a12da8-7e00-40c0-a736-3071a72c4147-kube-api-access-4jp84\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854247 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjql\" (UniqueName: \"kubernetes.io/projected/f753c88b-c4b9-4666-80b2-48a09074e5e8-kube-api-access-fhjql\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.854279 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.854339 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.354323493 +0000 UTC m=+35.600364277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854341 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7e1c9ab-363a-4900-ab06-01740f969943-config\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854351 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-config\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854455 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmvx9\" (UniqueName: \"kubernetes.io/projected/158e5a53-5c15-4402-8a11-7963c8246cfe-kube-api-access-cmvx9\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854486 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854510 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f5f068-c587-41ea-b827-a1736453bb8d-serving-cert\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e1c9ab-363a-4900-ab06-01740f969943-serving-cert\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854560 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60746a-01ad-4609-a3ef-e31bbdf74407-config-volume\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.854610 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xmnn\" (UniqueName: \"kubernetes.io/projected/e0677dda-7088-4f85-91d3-7d07b270b611-kube-api-access-9xmnn\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.854661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq4hm\" (UniqueName: \"kubernetes.io/projected/4207130c-c9e5-4358-9860-7f2ebffa6615-kube-api-access-sq4hm\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.854713 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.854727 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.854703 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.855000 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.354763108 +0000 UTC m=+35.600803917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.855026 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.355011004 +0000 UTC m=+35.601051792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f753c88b-c4b9-4666-80b2-48a09074e5e8-serving-cert\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.855333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q22h9\" (UniqueName: \"kubernetes.io/projected/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-kube-api-access-q22h9\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpzw\" (UniqueName: \"kubernetes.io/projected/939b0f33-4382-405b-ab91-17a162e03fbb-kube-api-access-mbpzw\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-default-certificate\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrsdw\" (UniqueName: \"kubernetes.io/projected/b914f895-ad9f-4472-88e3-2de946cf4a9f-kube-api-access-qrsdw\") pod \"network-check-source-7cbc45fddc-2kvhf\" (UID: \"b914f895-ad9f-4472-88e3-2de946cf4a9f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855439 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0677dda-7088-4f85-91d3-7d07b270b611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-trusted-ca\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85j98\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.855881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.855727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.857380 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.857296 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.857519 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.857400 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:34:45.857519 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.857457 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.357442604 +0000 UTC m=+35.603483405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.858286 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f5f068-c587-41ea-b827-a1736453bb8d-serving-cert\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.858784 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-stats-auth\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.858876 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.858965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f753c88b-c4b9-4666-80b2-48a09074e5e8-serving-cert\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.858991 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.859164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859013 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e1c9ab-363a-4900-ab06-01740f969943-serving-cert\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.859533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859158 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f753c88b-c4b9-4666-80b2-48a09074e5e8-trusted-ca\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.859533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859490 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-default-certificate\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.859533 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859526 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a12da8-7e00-40c0-a736-3071a72c4147-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.859667 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859528 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.859667 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.859587 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.860454 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.860437 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0677dda-7088-4f85-91d3-7d07b270b611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.864935 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.864913 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f5f068-c587-41ea-b827-a1736453bb8d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.870629 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.870585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9ctp\" (UniqueName: \"kubernetes.io/projected/47f5f068-c587-41ea-b827-a1736453bb8d-kube-api-access-t9ctp\") pod \"insights-operator-6f9c644dd9-spbxm\" (UID: \"47f5f068-c587-41ea-b827-a1736453bb8d\") " pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:45.870877 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.870750 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjzvd\" (UniqueName: \"kubernetes.io/projected/d7e1c9ab-363a-4900-ab06-01740f969943-kube-api-access-vjzvd\") pod \"service-ca-operator-57cdc4b768-k8zmc\" (UID: \"d7e1c9ab-363a-4900-ab06-01740f969943\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.873139 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.873090 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrsdw\" (UniqueName: \"kubernetes.io/projected/b914f895-ad9f-4472-88e3-2de946cf4a9f-kube-api-access-qrsdw\") pod \"network-check-source-7cbc45fddc-2kvhf\" (UID: \"b914f895-ad9f-4472-88e3-2de946cf4a9f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" Sep 17 17:34:45.873428 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.873277 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmvx9\" (UniqueName: \"kubernetes.io/projected/158e5a53-5c15-4402-8a11-7963c8246cfe-kube-api-access-cmvx9\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:45.873428 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.873348 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpzw\" (UniqueName: \"kubernetes.io/projected/939b0f33-4382-405b-ab91-17a162e03fbb-kube-api-access-mbpzw\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:45.874104 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.874080 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.874241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.874214 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xmnn\" (UniqueName: \"kubernetes.io/projected/e0677dda-7088-4f85-91d3-7d07b270b611-kube-api-access-9xmnn\") pod \"managed-serviceaccount-addon-agent-56b795b94b-5842r\" (UID: \"e0677dda-7088-4f85-91d3-7d07b270b611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:45.874241 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.874230 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq4hm\" (UniqueName: \"kubernetes.io/projected/4207130c-c9e5-4358-9860-7f2ebffa6615-kube-api-access-sq4hm\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:45.874847 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.874825 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjql\" (UniqueName: \"kubernetes.io/projected/f753c88b-c4b9-4666-80b2-48a09074e5e8-kube-api-access-fhjql\") pod \"console-operator-fd4bb46c-4qtmt\" (UID: \"f753c88b-c4b9-4666-80b2-48a09074e5e8\") " pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:45.875572 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.875553 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jp84\" (UniqueName: \"kubernetes.io/projected/46a12da8-7e00-40c0-a736-3071a72c4147-kube-api-access-4jp84\") pod \"kube-storage-version-migrator-operator-55889f7b47-4h7d8\" (UID: \"46a12da8-7e00-40c0-a736-3071a72c4147\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.875681 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.875664 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85j98\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:45.875963 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.875945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmtw8\" (UniqueName: \"kubernetes.io/projected/f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d-kube-api-access-zmtw8\") pod \"volume-data-source-validator-5c4db8bdf9-bxjrj\" (UID: \"f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" Sep 17 17:34:45.956374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q22h9\" (UniqueName: \"kubernetes.io/projected/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-kube-api-access-q22h9\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956424 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d162d8fa-063a-4f39-8d0c-c665350f1ec0-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv9kc\" (UniqueName: \"kubernetes.io/projected/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-kube-api-access-zv9kc\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956486 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td5jw\" (UniqueName: \"kubernetes.io/projected/e681489d-a82b-44a3-bade-59fe825f1260-kube-api-access-td5jw\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e681489d-a82b-44a3-bade-59fe825f1260-cert\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-tmp\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956772 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.956867 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.956942 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.45692214 +0000 UTC m=+35.702962951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.956975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b60746a-01ad-4609-a3ef-e31bbdf74407-tmp-dir\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.957010 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9b52\" (UniqueName: \"kubernetes.io/projected/2b60746a-01ad-4609-a3ef-e31bbdf74407-kube-api-access-g9b52\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-tmp\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957209 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d162d8fa-063a-4f39-8d0c-c665350f1ec0-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.957307 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:45.957377 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.457361122 +0000 UTC m=+35.703401911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.957740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60746a-01ad-4609-a3ef-e31bbdf74407-config-volume\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.958860 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957758 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b60746a-01ad-4609-a3ef-e31bbdf74407-tmp-dir\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.958860 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.957782 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.958860 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.958016 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" Sep 17 17:34:45.958860 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.958271 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60746a-01ad-4609-a3ef-e31bbdf74407-config-volume\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.959829 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.959722 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-ca\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.959948 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.959929 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e681489d-a82b-44a3-bade-59fe825f1260-cert\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.960003 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.959945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.960134 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.960095 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.960504 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.960485 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.961008 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.960992 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.964207 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.964189 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" Sep 17 17:34:45.966545 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.966524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q22h9\" (UniqueName: \"kubernetes.io/projected/5a1a6f91-02b2-4b27-ad17-75e0c9062ab0-kube-api-access-q22h9\") pod \"klusterlet-addon-workmgr-6b5967cb9-j4jp7\" (UID: \"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:45.966749 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.966728 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv9kc\" (UniqueName: \"kubernetes.io/projected/fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd-kube-api-access-zv9kc\") pod \"cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg\" (UID: \"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:45.966943 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.966925 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9b52\" (UniqueName: \"kubernetes.io/projected/2b60746a-01ad-4609-a3ef-e31bbdf74407-kube-api-access-g9b52\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:45.967042 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.966938 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td5jw\" (UniqueName: \"kubernetes.io/projected/e681489d-a82b-44a3-bade-59fe825f1260-kube-api-access-td5jw\") pod \"ingress-canary-qjmqs\" (UID: \"e681489d-a82b-44a3-bade-59fe825f1260\") " pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:45.981565 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.981533 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" Sep 17 17:34:45.993515 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:45.993485 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" Sep 17 17:34:46.011174 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.011147 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" Sep 17 17:34:46.017346 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.017306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:34:46.029392 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.029340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" Sep 17 17:34:46.048196 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.048159 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:34:46.054925 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.054865 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:34:46.071992 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.070758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qjmqs" Sep 17 17:34:46.221829 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.221801 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:34:46.223873 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.222270 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:34:46.227139 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.226374 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thgr8\"" Sep 17 17:34:46.227403 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.226705 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 17:34:46.299045 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.299021 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj"] Sep 17 17:34:46.302158 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.302130 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c474aa_358d_49fc_b2fd_c8f40d1c6e3d.slice/crio-7d4b6ed132046c3b3c27ff5edf82a3f95ff2b6a98ad0d445e1874d3da6bf550c WatchSource:0}: Error finding container 7d4b6ed132046c3b3c27ff5edf82a3f95ff2b6a98ad0d445e1874d3da6bf550c: Status 404 returned error can't find the container with id 7d4b6ed132046c3b3c27ff5edf82a3f95ff2b6a98ad0d445e1874d3da6bf550c Sep 17 17:34:46.302798 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.302785 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8"] Sep 17 17:34:46.304955 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.304901 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a12da8_7e00_40c0_a736_3071a72c4147.slice/crio-f8dc9733a3e7650c03e1b70b96fce8182ae956e3d8566f4202b406afc1ee1d94 WatchSource:0}: Error finding container f8dc9733a3e7650c03e1b70b96fce8182ae956e3d8566f4202b406afc1ee1d94: Status 404 returned error can't find the container with id f8dc9733a3e7650c03e1b70b96fce8182ae956e3d8566f4202b406afc1ee1d94 Sep 17 17:34:46.309571 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.309552 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc"] Sep 17 17:34:46.311657 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.311635 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e1c9ab_363a_4900_ab06_01740f969943.slice/crio-b10ae73d30ff563b6e778a5d46551f80886d4288a88cd8a7a729990217575395 WatchSource:0}: Error finding container b10ae73d30ff563b6e778a5d46551f80886d4288a88cd8a7a729990217575395: Status 404 returned error can't find the container with id b10ae73d30ff563b6e778a5d46551f80886d4288a88cd8a7a729990217575395 Sep 17 17:34:46.342225 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.342170 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-4qtmt"] Sep 17 17:34:46.344586 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.344563 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf753c88b_c4b9_4666_80b2_48a09074e5e8.slice/crio-27b970e88c1589ae8dcda88ec0b90cfe9f7c7fd116f72f182f0dfb5afd061493 WatchSource:0}: Error finding container 27b970e88c1589ae8dcda88ec0b90cfe9f7c7fd116f72f182f0dfb5afd061493: Status 404 returned error can't find the container with id 27b970e88c1589ae8dcda88ec0b90cfe9f7c7fd116f72f182f0dfb5afd061493 Sep 17 17:34:46.363885 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.363857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:46.364021 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.363907 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:46.364079 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364021 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:46.364079 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.364020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:46.364207 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364027 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:34:46.364207 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364097 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.364073539 +0000 UTC m=+36.610114339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:46.364207 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364151 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.364137183 +0000 UTC m=+36.610177972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:34:46.364207 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364171 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.36416229 +0000 UTC m=+36.610203073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:34:46.364411 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.364298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:46.364411 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.364361 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:46.364614 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364419 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:34:46.364614 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364471 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:34:46.364614 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364483 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:34:46.364614 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364471 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.364456034 +0000 UTC m=+36.610496840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:34:46.364614 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.364522 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.364511381 +0000 UTC m=+36.610552165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:34:46.389027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.388995 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-spbxm"] Sep 17 17:34:46.391574 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.391547 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r"] Sep 17 17:34:46.392228 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.392207 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f5f068_c587_41ea_b827_a1736453bb8d.slice/crio-ae950929c3576c549e24f7713315c56a4cdde7f7e249921119087cd78571c6b5 WatchSource:0}: Error finding container ae950929c3576c549e24f7713315c56a4cdde7f7e249921119087cd78571c6b5: Status 404 returned error can't find the container with id ae950929c3576c549e24f7713315c56a4cdde7f7e249921119087cd78571c6b5 Sep 17 17:34:46.394016 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.393938 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0677dda_7088_4f85_91d3_7d07b270b611.slice/crio-aedc2b643e01ad17dd9147427f219c4a898814d2328ab630aaf64f0c443e19c5 WatchSource:0}: Error finding container aedc2b643e01ad17dd9147427f219c4a898814d2328ab630aaf64f0c443e19c5: Status 404 returned error can't find the container with id aedc2b643e01ad17dd9147427f219c4a898814d2328ab630aaf64f0c443e19c5 Sep 17 17:34:46.416070 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.416038 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" event={"ID":"e0677dda-7088-4f85-91d3-7d07b270b611","Type":"ContainerStarted","Data":"aedc2b643e01ad17dd9147427f219c4a898814d2328ab630aaf64f0c443e19c5"} Sep 17 17:34:46.417211 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.417186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" event={"ID":"d7e1c9ab-363a-4900-ab06-01740f969943","Type":"ContainerStarted","Data":"b10ae73d30ff563b6e778a5d46551f80886d4288a88cd8a7a729990217575395"} Sep 17 17:34:46.420503 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.420473 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab89a7e9-37cb-46e9-8535-e1d6173de675" containerID="2338278d05534897e659fe7c5618ed60a70dc845e1e79d0fa1d29bc60f041caf" exitCode=0 Sep 17 17:34:46.420617 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.420555 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerDied","Data":"2338278d05534897e659fe7c5618ed60a70dc845e1e79d0fa1d29bc60f041caf"} Sep 17 17:34:46.422265 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.422236 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" event={"ID":"47f5f068-c587-41ea-b827-a1736453bb8d","Type":"ContainerStarted","Data":"ae950929c3576c549e24f7713315c56a4cdde7f7e249921119087cd78571c6b5"} Sep 17 17:34:46.422751 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.422702 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf"] Sep 17 17:34:46.423893 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.423872 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" event={"ID":"46a12da8-7e00-40c0-a736-3071a72c4147","Type":"ContainerStarted","Data":"f8dc9733a3e7650c03e1b70b96fce8182ae956e3d8566f4202b406afc1ee1d94"} Sep 17 17:34:46.425246 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.425222 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" event={"ID":"f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d","Type":"ContainerStarted","Data":"7d4b6ed132046c3b3c27ff5edf82a3f95ff2b6a98ad0d445e1874d3da6bf550c"} Sep 17 17:34:46.426460 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.426431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" event={"ID":"f753c88b-c4b9-4666-80b2-48a09074e5e8","Type":"ContainerStarted","Data":"27b970e88c1589ae8dcda88ec0b90cfe9f7c7fd116f72f182f0dfb5afd061493"} Sep 17 17:34:46.426630 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.426609 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb914f895_ad9f_4472_88e3_2de946cf4a9f.slice/crio-82e503cf266241eacd0b19fc96be7e58badac4595428b2c8ee7b86cf73810e44 WatchSource:0}: Error finding container 82e503cf266241eacd0b19fc96be7e58badac4595428b2c8ee7b86cf73810e44: Status 404 returned error can't find the container with id 82e503cf266241eacd0b19fc96be7e58badac4595428b2c8ee7b86cf73810e44 Sep 17 17:34:46.441895 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.441873 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qjmqs"] Sep 17 17:34:46.444403 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.444379 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode681489d_a82b_44a3_bade_59fe825f1260.slice/crio-61cffc2884f8c4c58d9984163bf897eae01ce709fa158c0c9b4af97f7bab2612 WatchSource:0}: Error finding container 61cffc2884f8c4c58d9984163bf897eae01ce709fa158c0c9b4af97f7bab2612: Status 404 returned error can't find the container with id 61cffc2884f8c4c58d9984163bf897eae01ce709fa158c0c9b4af97f7bab2612 Sep 17 17:34:46.462441 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.462420 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7"] Sep 17 17:34:46.464809 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.464781 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:46.464901 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.464833 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:46.464901 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.464785 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1a6f91_02b2_4b27_ad17_75e0c9062ab0.slice/crio-3355787a137d9ad96fac51302a09e372ae795fc98669fd988fdcb6113ee8961c WatchSource:0}: Error finding container 3355787a137d9ad96fac51302a09e372ae795fc98669fd988fdcb6113ee8961c: Status 404 returned error can't find the container with id 3355787a137d9ad96fac51302a09e372ae795fc98669fd988fdcb6113ee8961c Sep 17 17:34:46.464901 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.464877 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:34:46.465029 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.464986 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.464972619 +0000 UTC m=+36.711013401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:34:46.465073 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.465027 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:34:46.465108 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:46.465078 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.465062042 +0000 UTC m=+36.711102835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:34:46.512800 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:46.512774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg"] Sep 17 17:34:46.517067 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:34:46.517040 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd4ffba9_06db_49f7_86fb_fab0e7b4b3cd.slice/crio-a2131d84958337774e7fbce625935fe7d57b60ce5ddd9dd19e4b2c0cc58d16c0 WatchSource:0}: Error finding container a2131d84958337774e7fbce625935fe7d57b60ce5ddd9dd19e4b2c0cc58d16c0: Status 404 returned error can't find the container with id a2131d84958337774e7fbce625935fe7d57b60ce5ddd9dd19e4b2c0cc58d16c0 Sep 17 17:34:47.218236 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.218206 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:34:47.222464 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.222237 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwnj6\"" Sep 17 17:34:47.222464 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.222379 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 17:34:47.375103 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.375065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:47.375297 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.375170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:47.375297 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.375217 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:47.375404 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.375369 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:47.375448 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.375436 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.375416128 +0000 UTC m=+38.621456926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:47.375937 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.375907 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.375889676 +0000 UTC m=+38.621930492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:34:47.376044 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.375986 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:34:47.376044 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376023 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.37601129 +0000 UTC m=+38.622052072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:34:47.376180 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376086 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:34:47.376180 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376099 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:34:47.376180 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376151 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.376139653 +0000 UTC m=+38.622180451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:34:47.376180 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.376167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:47.376382 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.376253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:47.376382 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376344 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:34:47.376382 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.376378 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.376366873 +0000 UTC m=+38.622407671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:34:47.445051 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.444955 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" event={"ID":"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0","Type":"ContainerStarted","Data":"3355787a137d9ad96fac51302a09e372ae795fc98669fd988fdcb6113ee8961c"} Sep 17 17:34:47.452589 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.452495 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qjmqs" event={"ID":"e681489d-a82b-44a3-bade-59fe825f1260","Type":"ContainerStarted","Data":"61cffc2884f8c4c58d9984163bf897eae01ce709fa158c0c9b4af97f7bab2612"} Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.477279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.477372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.477686 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.477759 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.477739961 +0000 UTC m=+38.723780757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.478267 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:34:47.480047 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:47.478335 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.478317784 +0000 UTC m=+38.724358568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:34:47.493223 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.493099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44pd9" event={"ID":"ab89a7e9-37cb-46e9-8535-e1d6173de675","Type":"ContainerStarted","Data":"555ababd44b07bc43f373bb06c1983b93723e2bbe7b37b2fbe5dff4002d1654e"} Sep 17 17:34:47.499900 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.499844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerStarted","Data":"a2131d84958337774e7fbce625935fe7d57b60ce5ddd9dd19e4b2c0cc58d16c0"} Sep 17 17:34:47.506167 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:47.506075 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" event={"ID":"b914f895-ad9f-4472-88e3-2de946cf4a9f","Type":"ContainerStarted","Data":"82e503cf266241eacd0b19fc96be7e58badac4595428b2c8ee7b86cf73810e44"} Sep 17 17:34:49.401742 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.401695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.401803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.401896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.401949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.401995 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402187 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:49.402249 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402253 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.402233813 +0000 UTC m=+42.648274603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:49.402681 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402662 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:34:49.402761 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402716 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.402701679 +0000 UTC m=+42.648742467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:34:49.402828 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402784 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.402772897 +0000 UTC m=+42.648813700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:34:49.402889 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402845 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:34:49.402889 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402875 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.4028656 +0000 UTC m=+42.648906387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:34:49.402990 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402936 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:34:49.402990 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402947 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:34:49.402990 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.402978 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.402968586 +0000 UTC m=+42.649009372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:34:49.503172 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.503137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:49.503360 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:49.503212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:49.503489 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.503473 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:34:49.503570 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.503544 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.503523288 +0000 UTC m=+42.749564094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:34:49.503983 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.503939 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:34:49.504079 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:49.503993 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.503977202 +0000 UTC m=+42.750017991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:34:53.444831 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.444788 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.444867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.444913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.444937 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445006 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.444986438 +0000 UTC m=+50.691027230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445025 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445040 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445046 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.444945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445050 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.44503189 +0000 UTC m=+50.691072695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445139 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.445109806 +0000 UTC m=+50.691150589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445159 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.445152103 +0000 UTC m=+50.691192886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.445211 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445304 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:53.445323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.445338 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.445328241 +0000 UTC m=+50.691369028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:34:53.546187 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.546155 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:34:53.546187 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:34:53.546204 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:34:53.546430 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.546309 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:34:53.546430 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.546378 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.546359037 +0000 UTC m=+50.792399825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:34:53.546430 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.546317 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:34:53.546593 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:34:53.546477 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.546456119 +0000 UTC m=+50.792496937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:35:00.560831 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:00.560791 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" event={"ID":"f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d","Type":"ContainerStarted","Data":"b9880fc7f3387cbc86bfb3af055f973f2f159a3b4bd3a5eea2889204c5a1c785"} Sep 17 17:35:00.581164 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:00.581101 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bxjrj" podStartSLOduration=25.264909752 podStartE2EDuration="35.581087389s" totalImagesPullingTime="10.316177637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:00.580062871 +0000 UTC m=+49.826103675" watchObservedRunningTime="2026-09-17 17:35:00.581087389 +0000 UTC m=+49.827128190" Sep 17 17:35:00.581797 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:00.581771 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-44pd9" podStartSLOduration=21.033398833 podStartE2EDuration="49.581761881s" totalImagesPullingTime="28.548363048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:47.546954796 +0000 UTC m=+36.792995628" watchObservedRunningTime="2026-09-17 17:35:00.581761881 +0000 UTC m=+49.827802753" Sep 17 17:35:01.532202 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.532149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:01.532369 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.532220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:35:01.532369 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.532278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:01.532369 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.532342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:01.532551 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.532415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:35:01.532551 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532544 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:35:01.532654 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532605 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls podName:158e5a53-5c15-4402-8a11-7963c8246cfe nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.532586542 +0000 UTC m=+66.778627328 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-2zmkb" (UID: "158e5a53-5c15-4402-8a11-7963c8246cfe") : secret "samples-operator-tls" not found Sep 17 17:35:01.532716 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532672 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:35:01.532716 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532682 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb674dbf5-bnvhk: secret "image-registry-tls" not found Sep 17 17:35:01.532716 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532713 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls podName:d205cb9a-76b7-47e2-ae2c-e1fb60522128 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.532702117 +0000 UTC m=+66.778742915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls") pod "image-registry-6fb674dbf5-bnvhk" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128") : secret "image-registry-tls" not found Sep 17 17:35:01.532870 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532767 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:35:01.532870 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532797 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls podName:939b0f33-4382-405b-ab91-17a162e03fbb nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.532787841 +0000 UTC m=+66.778828626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-kzc24" (UID: "939b0f33-4382-405b-ab91-17a162e03fbb") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:35:01.532870 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532852 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:35:01.533020 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532878 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.532869618 +0000 UTC m=+66.778910403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : secret "router-metrics-certs-default" not found Sep 17 17:35:01.533020 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.532942 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle podName:4207130c-c9e5-4358-9860-7f2ebffa6615 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.532933226 +0000 UTC m=+66.778974011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle") pod "router-default-f58f99974-897ql" (UID: "4207130c-c9e5-4358-9860-7f2ebffa6615") : configmap references non-existent config key: service-ca.crt Sep 17 17:35:01.566586 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.566553 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" event={"ID":"47f5f068-c587-41ea-b827-a1736453bb8d","Type":"ContainerStarted","Data":"fe6a793ab87ddc9bf7d6a173d5f1f117950e5b64595d9793b232032c7ee1ab98"} Sep 17 17:35:01.568380 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.568351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" event={"ID":"46a12da8-7e00-40c0-a736-3071a72c4147","Type":"ContainerStarted","Data":"c6ba406120b67297808ddd28230b764a2d2d4c0ff9a26f98cda57d79b6310eae"} Sep 17 17:35:01.570819 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.570778 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerStarted","Data":"ee6583c388bede7d5cb007094e06ebc9ac33135725eeeabdb2f678a41322048f"} Sep 17 17:35:01.570936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.570835 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerStarted","Data":"c715e7d62118f9eacdd028afa7f0cf3ace372896b3c90b2d9098d7287bb44d77"} Sep 17 17:35:01.570936 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.570854 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerStarted","Data":"9625e4ee2407ea021541b305c1c06ae388b57119475ccd5dde75d2f26318a874"} Sep 17 17:35:01.572849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.572827 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" event={"ID":"b914f895-ad9f-4472-88e3-2de946cf4a9f","Type":"ContainerStarted","Data":"dd7257231a64a2ac52a38cdc7d7e23b9d3a2110fcb4aef4178ad2782ae39c84f"} Sep 17 17:35:01.576433 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.576397 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" event={"ID":"5a1a6f91-02b2-4b27-ad17-75e0c9062ab0","Type":"ContainerStarted","Data":"e4bf5ec7cf8409fa86a837349fb786b72bcadf00c3da8807039dcd04842bd85d"} Sep 17 17:35:01.577021 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.576649 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:35:01.578535 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.578513 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/0.log" Sep 17 17:35:01.578646 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.578555 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" Sep 17 17:35:01.578646 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.578591 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" event={"ID":"f753c88b-c4b9-4666-80b2-48a09074e5e8","Type":"ContainerDied","Data":"2fd9357266d7a98b1cb1873633ab4f5c0dd874eeff1f876de606e605c77514f8"} Sep 17 17:35:01.578646 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.578555 2583 generic.go:356] "Generic (PLEG): container finished" podID="f753c88b-c4b9-4666-80b2-48a09074e5e8" containerID="2fd9357266d7a98b1cb1873633ab4f5c0dd874eeff1f876de606e605c77514f8" exitCode=255 Sep 17 17:35:01.578806 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.578796 2583 scope.go:117] "RemoveContainer" containerID="2fd9357266d7a98b1cb1873633ab4f5c0dd874eeff1f876de606e605c77514f8" Sep 17 17:35:01.580798 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.580421 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qjmqs" event={"ID":"e681489d-a82b-44a3-bade-59fe825f1260","Type":"ContainerStarted","Data":"b677955ca9040aaab0b43cbdb6f679566baf855b04dae162da10d28c636c7583"} Sep 17 17:35:01.582103 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.582077 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" event={"ID":"e0677dda-7088-4f85-91d3-7d07b270b611","Type":"ContainerStarted","Data":"517eeeeda4ad0f6b74209af2292f4b64ab919c2fe695672ed82673867abd7d5b"} Sep 17 17:35:01.583697 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.583674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" event={"ID":"d7e1c9ab-363a-4900-ab06-01740f969943","Type":"ContainerStarted","Data":"e9db5d6dd6e0406e204c28e49ff43a88f6071f255ba5e2352144ced35d259d1e"} Sep 17 17:35:01.586395 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.586329 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" podStartSLOduration=22.729303216 podStartE2EDuration="36.586316696s" totalImagesPullingTime="13.85701348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.585515161 +0000 UTC m=+50.831555968" watchObservedRunningTime="2026-09-17 17:35:01.586316696 +0000 UTC m=+50.832357546" Sep 17 17:35:01.609136 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.607959 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56b795b94b-5842r" podStartSLOduration=27.345278518 podStartE2EDuration="41.607939195s" totalImagesPullingTime="14.262660677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.606344416 +0000 UTC m=+50.852385232" watchObservedRunningTime="2026-09-17 17:35:01.607939195 +0000 UTC m=+50.853980002" Sep 17 17:35:01.635151 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.633576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:01.635151 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.633683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:35:01.635151 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.634808 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:35:01.635151 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.635067 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert podName:d162d8fa-063a-4f39-8d0c-c665350f1ec0 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.634958256 +0000 UTC m=+66.880999044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-mcp9t" (UID: "d162d8fa-063a-4f39-8d0c-c665350f1ec0") : secret "networking-console-plugin-cert" not found Sep 17 17:35:01.637626 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.637444 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:35:01.637626 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:01.637527 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls podName:2b60746a-01ad-4609-a3ef-e31bbdf74407 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.637489476 +0000 UTC m=+66.883530258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls") pod "dns-default-bsbv6" (UID: "2b60746a-01ad-4609-a3ef-e31bbdf74407") : secret "dns-default-metrics-tls" not found Sep 17 17:35:01.647966 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.647913 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2kvhf" podStartSLOduration=19.772248877 podStartE2EDuration="33.647892678s" totalImagesPullingTime="13.875643801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.646690475 +0000 UTC m=+50.892731281" watchObservedRunningTime="2026-09-17 17:35:01.647892678 +0000 UTC m=+50.893933477" Sep 17 17:35:01.691168 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.690735 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" podStartSLOduration=22.732598086 podStartE2EDuration="36.690710685s" totalImagesPullingTime="13.958112599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.690140345 +0000 UTC m=+50.936181147" watchObservedRunningTime="2026-09-17 17:35:01.690710685 +0000 UTC m=+50.936751488" Sep 17 17:35:01.691495 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.691462 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b5967cb9-j4jp7" podStartSLOduration=27.855942980000002 podStartE2EDuration="41.691449213s" totalImagesPullingTime="13.835506233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.668149426 +0000 UTC m=+50.914190233" watchObservedRunningTime="2026-09-17 17:35:01.691449213 +0000 UTC m=+50.937490017" Sep 17 17:35:01.714569 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.714377 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" podStartSLOduration=27.910540659 podStartE2EDuration="41.714353069s" totalImagesPullingTime="13.80381241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.713152341 +0000 UTC m=+50.959193148" watchObservedRunningTime="2026-09-17 17:35:01.714353069 +0000 UTC m=+50.960393875" Sep 17 17:35:01.733851 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.733787 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" podStartSLOduration=22.789614963 podStartE2EDuration="36.733766389s" totalImagesPullingTime="13.944151426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.732584243 +0000 UTC m=+50.978625049" watchObservedRunningTime="2026-09-17 17:35:01.733766389 +0000 UTC m=+50.979807195" Sep 17 17:35:01.772555 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:01.771884 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qjmqs" podStartSLOduration=2.913574003 podStartE2EDuration="16.771860495s" totalImagesPullingTime="13.858286492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:01.770133773 +0000 UTC m=+51.016174575" watchObservedRunningTime="2026-09-17 17:35:01.771860495 +0000 UTC m=+51.017901301" Sep 17 17:35:02.505841 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.505804 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-224km"] Sep 17 17:35:02.508492 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.508470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.514977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.514541 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 17:35:02.514977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.514566 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2xjm4\"" Sep 17 17:35:02.514977 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.514632 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 17:35:02.525996 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.525968 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-224km"] Sep 17 17:35:02.588064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588036 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:35:02.588528 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/0.log" Sep 17 17:35:02.588582 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588547 2583 generic.go:356] "Generic (PLEG): container finished" podID="f753c88b-c4b9-4666-80b2-48a09074e5e8" containerID="8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8" exitCode=255 Sep 17 17:35:02.588698 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588670 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" event={"ID":"f753c88b-c4b9-4666-80b2-48a09074e5e8","Type":"ContainerDied","Data":"8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8"} Sep 17 17:35:02.588824 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588733 2583 scope.go:117] "RemoveContainer" containerID="2fd9357266d7a98b1cb1873633ab4f5c0dd874eeff1f876de606e605c77514f8" Sep 17 17:35:02.588947 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.588925 2583 scope.go:117] "RemoveContainer" containerID="8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8" Sep 17 17:35:02.589217 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:02.589163 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-4qtmt_openshift-console-operator(f753c88b-c4b9-4666-80b2-48a09074e5e8)\"" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" podUID="f753c88b-c4b9-4666-80b2-48a09074e5e8" Sep 17 17:35:02.646967 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.646932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-825kl\" (UniqueName: \"kubernetes.io/projected/54552639-5771-4e16-b7e4-23b4335e4faa-kube-api-access-825kl\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.647255 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.647229 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54552639-5771-4e16-b7e4-23b4335e4faa-data-volume\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.647518 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.647496 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54552639-5771-4e16-b7e4-23b4335e4faa-crio-socket\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.648094 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.648050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.648223 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.648151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.748915 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.748871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54552639-5771-4e16-b7e4-23b4335e4faa-crio-socket\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749095 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749095 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749057 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749248 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825kl\" (UniqueName: \"kubernetes.io/projected/54552639-5771-4e16-b7e4-23b4335e4faa-kube-api-access-825kl\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749248 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749206 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54552639-5771-4e16-b7e4-23b4335e4faa-data-volume\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749249 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54552639-5771-4e16-b7e4-23b4335e4faa-crio-socket\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.749405 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:02.749367 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:35:02.749450 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:02.749426 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:03.249405573 +0000 UTC m=+52.495446371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : configmap "kube-rbac-proxy" not found Sep 17 17:35:02.749545 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:02.749523 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:02.749666 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:02.749573 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:03.249560099 +0000 UTC m=+52.495600899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:02.749666 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.749632 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54552639-5771-4e16-b7e4-23b4335e4faa-data-volume\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:02.761847 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:02.761767 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-825kl\" (UniqueName: \"kubernetes.io/projected/54552639-5771-4e16-b7e4-23b4335e4faa-kube-api-access-825kl\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:03.253039 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:03.253002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:03.253039 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:03.253040 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:03.253268 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:03.253159 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:03.253268 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:03.253227 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:04.253213385 +0000 UTC m=+53.499254167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:03.253268 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:03.253251 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:35:03.253408 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:03.253308 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:04.253294319 +0000 UTC m=+53.499335126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : configmap "kube-rbac-proxy" not found Sep 17 17:35:03.594061 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:03.594031 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:35:03.594512 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:03.594492 2583 scope.go:117] "RemoveContainer" containerID="8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8" Sep 17 17:35:03.594723 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:03.594702 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-4qtmt_openshift-console-operator(f753c88b-c4b9-4666-80b2-48a09074e5e8)\"" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" podUID="f753c88b-c4b9-4666-80b2-48a09074e5e8" Sep 17 17:35:04.262700 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:04.262662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:04.262700 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:04.262705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:04.262926 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:04.262779 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:35:04.262926 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:04.262809 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:04.262926 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:04.262838 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:06.262824557 +0000 UTC m=+55.508865340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : configmap "kube-rbac-proxy" not found Sep 17 17:35:04.262926 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:04.262872 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:06.262849743 +0000 UTC m=+55.508890550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:05.050925 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:05.050897 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s9gfm_2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0/dns-node-resolver/0.log" Sep 17 17:35:06.017721 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.017688 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:35:06.017874 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.017747 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:35:06.018146 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.018109 2583 scope.go:117] "RemoveContainer" containerID="8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8" Sep 17 17:35:06.018323 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:06.018304 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-4qtmt_openshift-console-operator(f753c88b-c4b9-4666-80b2-48a09074e5e8)\"" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" podUID="f753c88b-c4b9-4666-80b2-48a09074e5e8" Sep 17 17:35:06.252415 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.252389 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97t8m_1f9ccc9b-6d34-4923-8320-ad4586f8f1d0/node-ca/0.log" Sep 17 17:35:06.280910 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.280813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:06.280910 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:06.280855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:06.281091 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:06.280944 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:35:06.281091 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:06.280981 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:06.281091 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:06.281061 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:10.281042905 +0000 UTC m=+59.527083688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:06.281227 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:06.281141 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:10.281106649 +0000 UTC m=+59.527147438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : configmap "kube-rbac-proxy" not found Sep 17 17:35:07.277724 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:07.277694 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qjmqs_e681489d-a82b-44a3-bade-59fe825f1260/serve-healthcheck-canary/0.log" Sep 17 17:35:07.854045 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:07.854014 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-4h7d8_46a12da8-7e00-40c0-a736-3071a72c4147/kube-storage-version-migrator-operator/0.log" Sep 17 17:35:10.318635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:10.318593 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:10.318635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:10.318632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:10.319062 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:10.318756 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:35:10.319062 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:10.318758 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:10.319062 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:10.318810 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:18.318797449 +0000 UTC m=+67.564838232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : configmap "kube-rbac-proxy" not found Sep 17 17:35:10.319062 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:10.318824 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:18.318817632 +0000 UTC m=+67.564858415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:10.405140 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:10.405100 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmrg9" Sep 17 17:35:16.983519 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.983475 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:35:16.984033 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.983582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:35:16.986504 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.986479 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 17:35:16.986629 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.986486 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 17:35:16.996635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.996605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26893c92-4fe2-4746-b1e0-43972e90d6de-original-pull-secret\") pod \"global-pull-secret-syncer-kw8q4\" (UID: \"26893c92-4fe2-4746-b1e0-43972e90d6de\") " pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:35:16.996635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:16.996631 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd47ebc-76f8-4ff4-92a6-39f2700c35be-metrics-certs\") pod \"network-metrics-daemon-nkskg\" (UID: \"cdd47ebc-76f8-4ff4-92a6-39f2700c35be\") " pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:35:17.084391 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.084353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:35:17.086874 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.086845 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt5dj\" (UniqueName: \"kubernetes.io/projected/51140b5c-ad9c-40c4-8e15-572191352895-kube-api-access-pt5dj\") pod \"network-check-target-lxqq9\" (UID: \"51140b5c-ad9c-40c4-8e15-572191352895\") " pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:35:17.162074 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.162035 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kw8q4" Sep 17 17:35:17.177843 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.177816 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thgr8\"" Sep 17 17:35:17.184923 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.184900 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:35:17.240685 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.240663 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwnj6\"" Sep 17 17:35:17.245038 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.245012 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkskg" Sep 17 17:35:17.450690 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.450650 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kw8q4"] Sep 17 17:35:17.453500 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:17.453474 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26893c92_4fe2_4746_b1e0_43972e90d6de.slice/crio-7ce9d0fa6cc58b635f2c871efa935943217b544a2f60304c9066c90002028a36 WatchSource:0}: Error finding container 7ce9d0fa6cc58b635f2c871efa935943217b544a2f60304c9066c90002028a36: Status 404 returned error can't find the container with id 7ce9d0fa6cc58b635f2c871efa935943217b544a2f60304c9066c90002028a36 Sep 17 17:35:17.471416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.471379 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lxqq9"] Sep 17 17:35:17.473934 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:17.473910 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51140b5c_ad9c_40c4_8e15_572191352895.slice/crio-0d6a5313e927c92f4fb031a7307f32be42099100ed8af16b20f18cbad228f600 WatchSource:0}: Error finding container 0d6a5313e927c92f4fb031a7307f32be42099100ed8af16b20f18cbad228f600: Status 404 returned error can't find the container with id 0d6a5313e927c92f4fb031a7307f32be42099100ed8af16b20f18cbad228f600 Sep 17 17:35:17.516140 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.516095 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nkskg"] Sep 17 17:35:17.520026 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:17.520003 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd47ebc_76f8_4ff4_92a6_39f2700c35be.slice/crio-c60175a295d85d923194487e59091ea4fdbe9afc7d34d046b7186bb423ee4a17 WatchSource:0}: Error finding container c60175a295d85d923194487e59091ea4fdbe9afc7d34d046b7186bb423ee4a17: Status 404 returned error can't find the container with id c60175a295d85d923194487e59091ea4fdbe9afc7d34d046b7186bb423ee4a17 Sep 17 17:35:17.589226 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589191 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:35:17.589405 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:17.589405 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:35:17.589405 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589331 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:17.589405 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589375 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:17.590042 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.589991 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4207130c-c9e5-4358-9860-7f2ebffa6615-service-ca-bundle\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:17.591778 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.591755 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4207130c-c9e5-4358-9860-7f2ebffa6615-metrics-certs\") pod \"router-default-f58f99974-897ql\" (UID: \"4207130c-c9e5-4358-9860-7f2ebffa6615\") " pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:17.591869 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.591759 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"image-registry-6fb674dbf5-bnvhk\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:17.592078 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.592060 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/158e5a53-5c15-4402-8a11-7963c8246cfe-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-2zmkb\" (UID: \"158e5a53-5c15-4402-8a11-7963c8246cfe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:35:17.592135 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.592081 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/939b0f33-4382-405b-ab91-17a162e03fbb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-kzc24\" (UID: \"939b0f33-4382-405b-ab91-17a162e03fbb\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:35:17.639159 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.639109 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkskg" event={"ID":"cdd47ebc-76f8-4ff4-92a6-39f2700c35be","Type":"ContainerStarted","Data":"c60175a295d85d923194487e59091ea4fdbe9afc7d34d046b7186bb423ee4a17"} Sep 17 17:35:17.640535 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.640514 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lxqq9" event={"ID":"51140b5c-ad9c-40c4-8e15-572191352895","Type":"ContainerStarted","Data":"c1f774b86044d65b0cc948572c5cf63faf6851c335711af00e8f1bdf71dcf1b8"} Sep 17 17:35:17.640626 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.640550 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lxqq9" event={"ID":"51140b5c-ad9c-40c4-8e15-572191352895","Type":"ContainerStarted","Data":"0d6a5313e927c92f4fb031a7307f32be42099100ed8af16b20f18cbad228f600"} Sep 17 17:35:17.640681 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.640632 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:35:17.641704 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.641683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kw8q4" event={"ID":"26893c92-4fe2-4746-b1e0-43972e90d6de","Type":"ContainerStarted","Data":"7ce9d0fa6cc58b635f2c871efa935943217b544a2f60304c9066c90002028a36"} Sep 17 17:35:17.661021 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.660967 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lxqq9" podStartSLOduration=66.660949899 podStartE2EDuration="1m6.660949899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:17.660605807 +0000 UTC m=+66.906646613" watchObservedRunningTime="2026-09-17 17:35:17.660949899 +0000 UTC m=+66.906990701" Sep 17 17:35:17.690062 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.690029 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:17.690215 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.690071 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:35:17.692358 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.692331 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b60746a-01ad-4609-a3ef-e31bbdf74407-metrics-tls\") pod \"dns-default-bsbv6\" (UID: \"2b60746a-01ad-4609-a3ef-e31bbdf74407\") " pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:17.692468 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.692426 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d162d8fa-063a-4f39-8d0c-c665350f1ec0-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-mcp9t\" (UID: \"d162d8fa-063a-4f39-8d0c-c665350f1ec0\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:35:17.751693 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.751617 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqbcm\"" Sep 17 17:35:17.759178 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.759151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:17.774631 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.774605 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hd9k5\"" Sep 17 17:35:17.782217 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.782188 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" Sep 17 17:35:17.791188 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.791155 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p22z4\"" Sep 17 17:35:17.799695 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.799663 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" Sep 17 17:35:17.827134 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.827055 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zlr65\"" Sep 17 17:35:17.835619 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.835571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:17.839697 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.839317 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-x5zgp\"" Sep 17 17:35:17.844935 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.844694 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdbm5\"" Sep 17 17:35:17.847570 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.846022 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" Sep 17 17:35:17.853129 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:17.852929 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:18.044319 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.044289 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:35:18.066932 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.066907 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24"] Sep 17 17:35:18.071283 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:18.071253 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939b0f33_4382_405b_ab91_17a162e03fbb.slice/crio-89df9635407e87de077758ab3594576c75c2ae965e58cddf210617d49065f407 WatchSource:0}: Error finding container 89df9635407e87de077758ab3594576c75c2ae965e58cddf210617d49065f407: Status 404 returned error can't find the container with id 89df9635407e87de077758ab3594576c75c2ae965e58cddf210617d49065f407 Sep 17 17:35:18.101644 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.100611 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb"] Sep 17 17:35:18.138376 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.138345 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f58f99974-897ql"] Sep 17 17:35:18.142416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.142390 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t"] Sep 17 17:35:18.146455 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:18.146376 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4207130c_c9e5_4358_9860_7f2ebffa6615.slice/crio-f3f3fd8ec2928318f1debc7f3b33683bde7fc414eff1bd8c4dbbaa0398100377 WatchSource:0}: Error finding container f3f3fd8ec2928318f1debc7f3b33683bde7fc414eff1bd8c4dbbaa0398100377: Status 404 returned error can't find the container with id f3f3fd8ec2928318f1debc7f3b33683bde7fc414eff1bd8c4dbbaa0398100377 Sep 17 17:35:18.176523 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.176466 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bsbv6"] Sep 17 17:35:18.180108 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:18.180080 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b60746a_01ad_4609_a3ef_e31bbdf74407.slice/crio-ddbe7c360f0408c187d55dbe7058adc18ab73351b89f6051e79c29b35f48f16b WatchSource:0}: Error finding container ddbe7c360f0408c187d55dbe7058adc18ab73351b89f6051e79c29b35f48f16b: Status 404 returned error can't find the container with id ddbe7c360f0408c187d55dbe7058adc18ab73351b89f6051e79c29b35f48f16b Sep 17 17:35:18.397066 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.395698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:18.397066 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.395750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:18.397066 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.396370 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54552639-5771-4e16-b7e4-23b4335e4faa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:18.397066 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:18.396494 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:35:18.397066 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:35:18.396550 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls podName:54552639-5771-4e16-b7e4-23b4335e4faa nodeName:}" failed. No retries permitted until 2026-09-17 17:35:34.396533437 +0000 UTC m=+83.642574236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-224km" (UID: "54552639-5771-4e16-b7e4-23b4335e4faa") : secret "insights-runtime-extractor-tls" not found Sep 17 17:35:18.648812 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.648618 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f58f99974-897ql" event={"ID":"4207130c-c9e5-4358-9860-7f2ebffa6615","Type":"ContainerStarted","Data":"f13b4012bfbfafb7db7e4183a35c3e22376e4b3c5bb2a67b046b390c1bae0cd2"} Sep 17 17:35:18.648812 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.648726 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f58f99974-897ql" event={"ID":"4207130c-c9e5-4358-9860-7f2ebffa6615","Type":"ContainerStarted","Data":"f3f3fd8ec2928318f1debc7f3b33683bde7fc414eff1bd8c4dbbaa0398100377"} Sep 17 17:35:18.651297 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.651265 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" event={"ID":"158e5a53-5c15-4402-8a11-7963c8246cfe","Type":"ContainerStarted","Data":"032468c8a5de2ebfd30d3a6a61368061bd9745afa00aa98b3dc0469901c7cac0"} Sep 17 17:35:18.654187 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.654147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" event={"ID":"939b0f33-4382-405b-ab91-17a162e03fbb","Type":"ContainerStarted","Data":"89df9635407e87de077758ab3594576c75c2ae965e58cddf210617d49065f407"} Sep 17 17:35:18.657227 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.657193 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" event={"ID":"d205cb9a-76b7-47e2-ae2c-e1fb60522128","Type":"ContainerStarted","Data":"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95"} Sep 17 17:35:18.657343 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.657248 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" event={"ID":"d205cb9a-76b7-47e2-ae2c-e1fb60522128","Type":"ContainerStarted","Data":"130f57e25b43dc91bea024b7f9df1aa46a8c770c4e681334e7cdb675e67726a0"} Sep 17 17:35:18.657887 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.657824 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:18.659990 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.659957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsbv6" event={"ID":"2b60746a-01ad-4609-a3ef-e31bbdf74407","Type":"ContainerStarted","Data":"ddbe7c360f0408c187d55dbe7058adc18ab73351b89f6051e79c29b35f48f16b"} Sep 17 17:35:18.662074 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.662054 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" event={"ID":"d162d8fa-063a-4f39-8d0c-c665350f1ec0","Type":"ContainerStarted","Data":"6cbabc5af3721a81995ce034413234e93a32393b174a2ced4b453ba65b104df4"} Sep 17 17:35:18.673861 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.672868 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-f58f99974-897ql" podStartSLOduration=53.672848205 podStartE2EDuration="53.672848205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:18.672035945 +0000 UTC m=+67.918076765" watchObservedRunningTime="2026-09-17 17:35:18.672848205 +0000 UTC m=+67.918889008" Sep 17 17:35:18.703366 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.701929 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" podStartSLOduration=67.701905083 podStartE2EDuration="1m7.701905083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:18.700208599 +0000 UTC m=+67.946249406" watchObservedRunningTime="2026-09-17 17:35:18.701905083 +0000 UTC m=+67.947945888" Sep 17 17:35:18.836363 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.836330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:18.839374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:18.839099 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:19.673235 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:19.672611 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkskg" event={"ID":"cdd47ebc-76f8-4ff4-92a6-39f2700c35be","Type":"ContainerStarted","Data":"8838870f61bc9bf829128fb3f33b1620f4743a3d03a4714ac764b364ebb3a8b0"} Sep 17 17:35:19.673235 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:19.672674 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:19.673235 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:19.672700 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkskg" event={"ID":"cdd47ebc-76f8-4ff4-92a6-39f2700c35be","Type":"ContainerStarted","Data":"364b3dec38474e2c39b5d71ff793e81b2efbcf75ecc1ff6e2354e4c208e39dc3"} Sep 17 17:35:19.675681 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:19.675656 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-f58f99974-897ql" Sep 17 17:35:19.699635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:19.699571 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nkskg" podStartSLOduration=67.658083729 podStartE2EDuration="1m8.699548657s" totalImagesPullingTime="1.041464928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:19.698069675 +0000 UTC m=+68.944110506" watchObservedRunningTime="2026-09-17 17:35:19.699548657 +0000 UTC m=+68.945589463" Sep 17 17:35:21.220155 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:21.220106 2583 scope.go:117] "RemoveContainer" containerID="8cef26ed59375327ff263fb402c1ed3b6ef3938e07db8742de0b16708f0732a8" Sep 17 17:35:23.689546 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.689442 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" event={"ID":"158e5a53-5c15-4402-8a11-7963c8246cfe","Type":"ContainerStarted","Data":"4fe7ed5501c018a506d7977ba2955c9436b734a9e6c1c908fdf9968480028b80"} Sep 17 17:35:23.689546 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.689517 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" event={"ID":"158e5a53-5c15-4402-8a11-7963c8246cfe","Type":"ContainerStarted","Data":"7ab40c3ad694ea47509862a6c2c262154fa17d0acba6c79dd693c57d9b6e76d9"} Sep 17 17:35:23.691129 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.691086 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" event={"ID":"939b0f33-4382-405b-ab91-17a162e03fbb","Type":"ContainerStarted","Data":"80b608bb8e9510bc92a8d0bd13323bc573792ddd8fbd34a4e672acb7e113b277"} Sep 17 17:35:23.692927 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.692900 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsbv6" event={"ID":"2b60746a-01ad-4609-a3ef-e31bbdf74407","Type":"ContainerStarted","Data":"b984df31bea8385948e35552b836389c99878937047c137b246abe0f0b750ca4"} Sep 17 17:35:23.693039 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.692947 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsbv6" event={"ID":"2b60746a-01ad-4609-a3ef-e31bbdf74407","Type":"ContainerStarted","Data":"fbf86d9bf91123ed0a247d564e65fa9fd8f77ccf88bb9e8e5b8902b33c4c10a1"} Sep 17 17:35:23.693103 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.693039 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:23.694516 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.694496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kw8q4" event={"ID":"26893c92-4fe2-4746-b1e0-43972e90d6de","Type":"ContainerStarted","Data":"2e9cd269de637b39e7f8665b5855f2959cabb2f8c4ed094f986243b08bcad94e"} Sep 17 17:35:23.696388 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.696372 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:35:23.696483 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.696446 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" event={"ID":"f753c88b-c4b9-4666-80b2-48a09074e5e8","Type":"ContainerStarted","Data":"7858aa011d042844424eeebe9526ebba7970da21f0c2f22b9515aed66b08cfda"} Sep 17 17:35:23.696666 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.696652 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:35:23.698140 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.698100 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" event={"ID":"d162d8fa-063a-4f39-8d0c-c665350f1ec0","Type":"ContainerStarted","Data":"80944bdb0130cc4d8a08c4c561110fb6344126d6ff9ce6f76263a68d93ee19a5"} Sep 17 17:35:23.709363 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.709313 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-2zmkb" podStartSLOduration=53.991141106 podStartE2EDuration="58.709298038s" totalImagesPullingTime="4.718156932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.708925308 +0000 UTC m=+72.954966114" watchObservedRunningTime="2026-09-17 17:35:23.709298038 +0000 UTC m=+72.955338842" Sep 17 17:35:23.729659 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.729610 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-mcp9t" podStartSLOduration=46.007264432 podStartE2EDuration="50.729594444s" totalImagesPullingTime="4.722330012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.728269001 +0000 UTC m=+72.974309808" watchObservedRunningTime="2026-09-17 17:35:23.729594444 +0000 UTC m=+72.975635249" Sep 17 17:35:23.749317 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.749268 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" podStartSLOduration=44.822104678 podStartE2EDuration="58.749245325s" totalImagesPullingTime="13.927140647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.748156369 +0000 UTC m=+72.994197174" watchObservedRunningTime="2026-09-17 17:35:23.749245325 +0000 UTC m=+72.995286129" Sep 17 17:35:23.774327 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.773646 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bsbv6" podStartSLOduration=34.075354438 podStartE2EDuration="38.773626716s" totalImagesPullingTime="4.698272278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.773386216 +0000 UTC m=+73.019427025" watchObservedRunningTime="2026-09-17 17:35:23.773626716 +0000 UTC m=+73.019667522" Sep 17 17:35:23.808524 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.808460 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kw8q4" podStartSLOduration=67.117380062 podStartE2EDuration="1m12.80843965s" totalImagesPullingTime="5.691059588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.791598169 +0000 UTC m=+73.037638977" watchObservedRunningTime="2026-09-17 17:35:23.80843965 +0000 UTC m=+73.054480456" Sep 17 17:35:23.809355 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.809322 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-kzc24" podStartSLOduration=54.007182185 podStartE2EDuration="58.809310019s" totalImagesPullingTime="4.802127834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.80804516 +0000 UTC m=+73.054085966" watchObservedRunningTime="2026-09-17 17:35:23.809310019 +0000 UTC m=+73.055350823" Sep 17 17:35:23.989452 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:23.989372 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-4qtmt" Sep 17 17:35:33.703813 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:33.703780 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bsbv6" Sep 17 17:35:34.429484 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:34.429441 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:34.431881 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:34.431858 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54552639-5771-4e16-b7e4-23b4335e4faa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-224km\" (UID: \"54552639-5771-4e16-b7e4-23b4335e4faa\") " pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:34.620036 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:34.620002 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2xjm4\"" Sep 17 17:35:34.628543 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:34.628506 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-224km" Sep 17 17:35:34.903755 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:34.903729 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-224km"] Sep 17 17:35:34.906367 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:34.906329 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54552639_5771_4e16_b7e4_23b4335e4faa.slice/crio-6c5d71f8ea739634b70f93c05f7bb68bf7b7fe028425e6ad7862df58fb3999cd WatchSource:0}: Error finding container 6c5d71f8ea739634b70f93c05f7bb68bf7b7fe028425e6ad7862df58fb3999cd: Status 404 returned error can't find the container with id 6c5d71f8ea739634b70f93c05f7bb68bf7b7fe028425e6ad7862df58fb3999cd Sep 17 17:35:35.744669 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:35.744630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-224km" event={"ID":"54552639-5771-4e16-b7e4-23b4335e4faa","Type":"ContainerStarted","Data":"8e9e5dcbe07e98a9274f81bdd35166a1d5a9fa336965b3d0c2d5ee9e9aafb818"} Sep 17 17:35:35.744839 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:35.744694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-224km" event={"ID":"54552639-5771-4e16-b7e4-23b4335e4faa","Type":"ContainerStarted","Data":"ea2c6fa7797c116db65620c1dbe764a00561cfaeda240206bb4bf9ef05fff267"} Sep 17 17:35:35.744839 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:35.744717 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-224km" event={"ID":"54552639-5771-4e16-b7e4-23b4335e4faa","Type":"ContainerStarted","Data":"6c5d71f8ea739634b70f93c05f7bb68bf7b7fe028425e6ad7862df58fb3999cd"} Sep 17 17:35:37.755201 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:37.755154 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-224km" event={"ID":"54552639-5771-4e16-b7e4-23b4335e4faa","Type":"ContainerStarted","Data":"a9abdb1301423cfdf4f3e4694d136dad97c755e8fe2ed56a789b0da97c99fa49"} Sep 17 17:35:37.763512 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:37.763471 2583 patch_prober.go:28] interesting pod/image-registry-6fb674dbf5-bnvhk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 17:35:37.763659 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:37.763533 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:35:37.780382 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:37.780317 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-224km" podStartSLOduration=34.015872396 podStartE2EDuration="35.780295873s" totalImagesPullingTime="1.764423477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:35:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:37.78015294 +0000 UTC m=+87.026193746" watchObservedRunningTime="2026-09-17 17:35:37.780295873 +0000 UTC m=+87.026336678" Sep 17 17:35:40.678893 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:40.678861 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:35:43.156878 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.156834 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jmk6p"] Sep 17 17:35:43.159849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.159826 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.168937 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.168914 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vptsg\"" Sep 17 17:35:43.169061 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.169002 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 17:35:43.169910 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.169893 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 17:35:43.171460 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.171441 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 17:35:43.171668 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.171652 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 17:35:43.199132 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199076 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-sys\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199132 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-textfile\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199177 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-accelerators-collector-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-tls\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-metrics-client-ca\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199286 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc9g9\" (UniqueName: \"kubernetes.io/projected/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-kube-api-access-lc9g9\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-wtmp\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.199348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.199332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-root\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300199 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300211 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-accelerators-collector-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-tls\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-metrics-client-ca\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300305 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc9g9\" (UniqueName: \"kubernetes.io/projected/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-kube-api-access-lc9g9\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300606 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-wtmp\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300606 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-root\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300606 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-sys\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-textfile\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300633 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-wtmp\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-root\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300689 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-sys\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300968 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300942 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-accelerators-collector-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.300968 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.300956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-textfile\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.301163 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.301140 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-metrics-client-ca\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.302684 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.302660 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-tls\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.302795 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.302739 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.325731 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.325698 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc9g9\" (UniqueName: \"kubernetes.io/projected/e11cfd9c-1afd-4c95-81b0-3fbb07a9d370-kube-api-access-lc9g9\") pod \"node-exporter-jmk6p\" (UID: \"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370\") " pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.469007 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.468915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jmk6p" Sep 17 17:35:43.480796 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:35:43.480756 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11cfd9c_1afd_4c95_81b0_3fbb07a9d370.slice/crio-6ae278063458146de0bbcc3507e5e2034a40a40af859e4f46291ad6962c4adaf WatchSource:0}: Error finding container 6ae278063458146de0bbcc3507e5e2034a40a40af859e4f46291ad6962c4adaf: Status 404 returned error can't find the container with id 6ae278063458146de0bbcc3507e5e2034a40a40af859e4f46291ad6962c4adaf Sep 17 17:35:43.780821 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:43.780786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jmk6p" event={"ID":"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370","Type":"ContainerStarted","Data":"6ae278063458146de0bbcc3507e5e2034a40a40af859e4f46291ad6962c4adaf"} Sep 17 17:35:44.787362 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:44.787264 2583 generic.go:356] "Generic (PLEG): container finished" podID="e11cfd9c-1afd-4c95-81b0-3fbb07a9d370" containerID="b345a659bcce11e9a0bd9524ece9804aafeac9819df0be0f0c3043f4d2bfb25f" exitCode=0 Sep 17 17:35:44.787719 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:44.787352 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jmk6p" event={"ID":"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370","Type":"ContainerDied","Data":"b345a659bcce11e9a0bd9524ece9804aafeac9819df0be0f0c3043f4d2bfb25f"} Sep 17 17:35:45.793218 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:45.793182 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jmk6p" event={"ID":"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370","Type":"ContainerStarted","Data":"f4ae2ed0e985c828ea75f07232268b76caee52c9180e92d2a217a031781ef115"} Sep 17 17:35:45.793606 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:45.793237 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jmk6p" event={"ID":"e11cfd9c-1afd-4c95-81b0-3fbb07a9d370","Type":"ContainerStarted","Data":"56a74744c1cc02ae91fd8b60e58bc8097123e22d87c1fc2561bf1f2e5f914fb0"} Sep 17 17:35:45.823898 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:45.823847 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jmk6p" podStartSLOduration=1.718933057 podStartE2EDuration="2.823828145s" totalImagesPullingTime="1.040202993s" totalInitContainerRuntime="64.692095ms" isStatefulPod=true podCreationTimestamp="2026-09-17 17:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:45.821986766 +0000 UTC m=+95.068027570" watchObservedRunningTime="2026-09-17 17:35:45.823828145 +0000 UTC m=+95.069868950" Sep 17 17:35:48.665278 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:48.665238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lxqq9" Sep 17 17:35:53.258979 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:35:53.258941 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:36:06.876691 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:06.876604 2583 generic.go:356] "Generic (PLEG): container finished" podID="d7e1c9ab-363a-4900-ab06-01740f969943" containerID="e9db5d6dd6e0406e204c28e49ff43a88f6071f255ba5e2352144ced35d259d1e" exitCode=0 Sep 17 17:36:06.876691 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:06.876675 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" event={"ID":"d7e1c9ab-363a-4900-ab06-01740f969943","Type":"ContainerDied","Data":"e9db5d6dd6e0406e204c28e49ff43a88f6071f255ba5e2352144ced35d259d1e"} Sep 17 17:36:06.877079 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:06.876998 2583 scope.go:117] "RemoveContainer" containerID="e9db5d6dd6e0406e204c28e49ff43a88f6071f255ba5e2352144ced35d259d1e" Sep 17 17:36:07.882899 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:07.882859 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-k8zmc" event={"ID":"d7e1c9ab-363a-4900-ab06-01740f969943","Type":"ContainerStarted","Data":"74b13f8dbc4be9ef3ace973b1d60d5c41bd244b3b5f6560aefd3619d92bb63dc"} Sep 17 17:36:16.917843 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:16.917806 2583 generic.go:356] "Generic (PLEG): container finished" podID="46a12da8-7e00-40c0-a736-3071a72c4147" containerID="c6ba406120b67297808ddd28230b764a2d2d4c0ff9a26f98cda57d79b6310eae" exitCode=0 Sep 17 17:36:16.918263 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:16.917868 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" event={"ID":"46a12da8-7e00-40c0-a736-3071a72c4147","Type":"ContainerDied","Data":"c6ba406120b67297808ddd28230b764a2d2d4c0ff9a26f98cda57d79b6310eae"} Sep 17 17:36:16.918263 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:16.918225 2583 scope.go:117] "RemoveContainer" containerID="c6ba406120b67297808ddd28230b764a2d2d4c0ff9a26f98cda57d79b6310eae" Sep 17 17:36:17.923810 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:17.923780 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-4h7d8" event={"ID":"46a12da8-7e00-40c0-a736-3071a72c4147","Type":"ContainerStarted","Data":"58e42234f7f96ffce4a109a848e9ae29492a34aa143248b2f60ffa0633bd37bf"} Sep 17 17:36:18.278455 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.278409 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerName="registry" containerID="cri-o://60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95" gracePeriod=30 Sep 17 17:36:18.513360 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.513335 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:36:18.615618 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615527 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615618 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615570 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615618 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615591 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85j98\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615618 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615619 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615954 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615658 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615954 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615679 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615954 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615714 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.615954 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.615745 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted\") pod \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\" (UID: \"d205cb9a-76b7-47e2-ae2c-e1fb60522128\") " Sep 17 17:36:18.616416 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.616337 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:36:18.616544 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.616408 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:36:18.618271 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.618233 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:36:18.618375 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.618345 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98" (OuterVolumeSpecName: "kube-api-access-85j98") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "kube-api-access-85j98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:36:18.618428 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.618383 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:36:18.618470 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.618446 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:36:18.618681 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.618656 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:36:18.626644 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.626610 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d205cb9a-76b7-47e2-ae2c-e1fb60522128" (UID: "d205cb9a-76b7-47e2-ae2c-e1fb60522128"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:36:18.717008 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.716968 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-certificates\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717008 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717001 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d205cb9a-76b7-47e2-ae2c-e1fb60522128-ca-trust-extracted\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717008 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717015 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-bound-sa-token\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717287 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717028 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-installation-pull-secrets\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717287 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717041 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85j98\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-kube-api-access-85j98\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717287 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717052 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d205cb9a-76b7-47e2-ae2c-e1fb60522128-registry-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717287 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717064 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d205cb9a-76b7-47e2-ae2c-e1fb60522128-image-registry-private-configuration\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.717287 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.717076 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d205cb9a-76b7-47e2-ae2c-e1fb60522128-trusted-ca\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:36:18.929187 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.929087 2583 generic.go:356] "Generic (PLEG): container finished" podID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerID="60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95" exitCode=0 Sep 17 17:36:18.929187 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.929162 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" Sep 17 17:36:18.929706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.929183 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" event={"ID":"d205cb9a-76b7-47e2-ae2c-e1fb60522128","Type":"ContainerDied","Data":"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95"} Sep 17 17:36:18.929706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.929284 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb674dbf5-bnvhk" event={"ID":"d205cb9a-76b7-47e2-ae2c-e1fb60522128","Type":"ContainerDied","Data":"130f57e25b43dc91bea024b7f9df1aa46a8c770c4e681334e7cdb675e67726a0"} Sep 17 17:36:18.929706 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.929320 2583 scope.go:117] "RemoveContainer" containerID="60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95" Sep 17 17:36:18.938333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.938315 2583 scope.go:117] "RemoveContainer" containerID="60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95" Sep 17 17:36:18.938599 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:36:18.938578 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95\": container with ID starting with 60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95 not found: ID does not exist" containerID="60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95" Sep 17 17:36:18.938657 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.938605 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95"} err="failed to get container status \"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95\": rpc error: code = NotFound desc = could not find container \"60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95\": container with ID starting with 60b856a845b5f01f98494e35cee113c8ba27b60386eaeaf1c13eca354d277d95 not found: ID does not exist" Sep 17 17:36:18.955246 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.955216 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:36:18.962681 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:18.962647 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fb674dbf5-bnvhk"] Sep 17 17:36:19.223425 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:19.223345 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" path="/var/lib/kubelet/pods/d205cb9a-76b7-47e2-ae2c-e1fb60522128/volumes" Sep 17 17:36:26.055942 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:26.055898 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" podUID="fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 17:36:26.961925 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:26.961890 2583 generic.go:356] "Generic (PLEG): container finished" podID="47f5f068-c587-41ea-b827-a1736453bb8d" containerID="fe6a793ab87ddc9bf7d6a173d5f1f117950e5b64595d9793b232032c7ee1ab98" exitCode=0 Sep 17 17:36:26.962095 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:26.961963 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" event={"ID":"47f5f068-c587-41ea-b827-a1736453bb8d","Type":"ContainerDied","Data":"fe6a793ab87ddc9bf7d6a173d5f1f117950e5b64595d9793b232032c7ee1ab98"} Sep 17 17:36:26.962331 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:26.962318 2583 scope.go:117] "RemoveContainer" containerID="fe6a793ab87ddc9bf7d6a173d5f1f117950e5b64595d9793b232032c7ee1ab98" Sep 17 17:36:27.967503 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:27.967463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-spbxm" event={"ID":"47f5f068-c587-41ea-b827-a1736453bb8d","Type":"ContainerStarted","Data":"d6113e828a1ac872a3c8679ba17778b576b06087b37af2ec61d9329e85bc1bf8"} Sep 17 17:36:36.056515 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:36.056441 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" podUID="fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 17:36:46.056553 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:46.056511 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" podUID="fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 17:36:46.057048 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:46.056601 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" Sep 17 17:36:46.057280 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:46.057257 2583 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ee6583c388bede7d5cb007094e06ebc9ac33135725eeeabdb2f678a41322048f"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 17 17:36:46.057381 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:46.057313 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" podUID="fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd" containerName="service-proxy" containerID="cri-o://ee6583c388bede7d5cb007094e06ebc9ac33135725eeeabdb2f678a41322048f" gracePeriod=30 Sep 17 17:36:47.048475 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:47.048441 2583 generic.go:356] "Generic (PLEG): container finished" podID="fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd" containerID="ee6583c388bede7d5cb007094e06ebc9ac33135725eeeabdb2f678a41322048f" exitCode=2 Sep 17 17:36:47.048622 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:47.048515 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerDied","Data":"ee6583c388bede7d5cb007094e06ebc9ac33135725eeeabdb2f678a41322048f"} Sep 17 17:36:47.048622 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:36:47.048559 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f5d7bbbf6-nkvmg" event={"ID":"fd4ffba9-06db-49f7-86fb-fab0e7b4b3cd","Type":"ContainerStarted","Data":"53a5e717ee04ee5a2079dd156a4fda9b2d93f50627595349800690cf80865ce0"} Sep 17 17:39:11.087448 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:39:11.087420 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:39:11.091869 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:39:11.091843 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:39:11.099347 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:39:11.099321 2583 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 17:44:11.121064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:44:11.121034 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:44:11.121635 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:44:11.121228 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:45:39.229612 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.229578 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:45:39.229995 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.229827 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerName="registry" Sep 17 17:45:39.229995 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.229837 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerName="registry" Sep 17 17:45:39.229995 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.229897 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d205cb9a-76b7-47e2-ae2c-e1fb60522128" containerName="registry" Sep 17 17:45:39.232529 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.232514 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.235151 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.235127 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5b8c1-kube-rbac-proxy-sar-config\"" Sep 17 17:45:39.235267 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.235153 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-28pvs\"" Sep 17 17:45:39.235267 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.235196 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5b8c1-serving-cert\"" Sep 17 17:45:39.236000 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.235985 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:45:39.246849 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.246827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:45:39.319988 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.319965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.320090 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.320024 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.421463 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.421436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.421587 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.421474 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.422069 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.422048 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.423981 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.423960 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls\") pod \"switch-graph-5b8c1-7f5b78944d-ngml2\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.542013 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.541987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:39.814949 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.814926 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:45:39.816695 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:45:39.816670 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5664c0_32ef_4d2a_a242_53ecdb80cfa3.slice/crio-e25d9d446685a42235880c10f8bb9002c0fe63b5c894bddc663fb95635c85d80 WatchSource:0}: Error finding container e25d9d446685a42235880c10f8bb9002c0fe63b5c894bddc663fb95635c85d80: Status 404 returned error can't find the container with id e25d9d446685a42235880c10f8bb9002c0fe63b5c894bddc663fb95635c85d80 Sep 17 17:45:39.818585 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:39.818564 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 17:45:40.057066 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:40.057033 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" event={"ID":"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3","Type":"ContainerStarted","Data":"e25d9d446685a42235880c10f8bb9002c0fe63b5c894bddc663fb95635c85d80"} Sep 17 17:45:43.073808 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:43.073770 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" event={"ID":"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3","Type":"ContainerStarted","Data":"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea"} Sep 17 17:45:43.074321 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:43.073907 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:49.083994 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:49.083953 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:45:49.102209 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:49.102147 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podStartSLOduration=7.779061631 podStartE2EDuration="10.102105958s" totalImagesPullingTime="2.323044327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:45:43.102359575 +0000 UTC m=+692.348400398" watchObservedRunningTime="2026-09-17 17:45:49.102105958 +0000 UTC m=+698.348146832" Sep 17 17:45:49.445567 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:49.445486 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:45:49.445727 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:49.445679 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" containerID="cri-o://0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea" gracePeriod=30 Sep 17 17:45:54.082378 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:54.082329 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:45:59.082650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:45:59.082609 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:04.082220 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:04.082139 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:04.082555 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:04.082287 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:46:09.082940 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:09.082902 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:14.082619 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:14.082571 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:19.082340 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.082298 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:19.587152 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.587127 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:46:19.700525 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.700454 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle\") pod \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " Sep 17 17:46:19.700525 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.700530 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls\") pod \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\" (UID: \"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3\") " Sep 17 17:46:19.700866 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.700839 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" (UID: "7f5664c0-32ef-4d2a-a242-53ecdb80cfa3"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:46:19.702657 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.702641 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" (UID: "7f5664c0-32ef-4d2a-a242-53ecdb80cfa3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:46:19.801818 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.801795 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:46:19.801818 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:19.801817 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:46:20.215600 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.215567 2583 generic.go:356] "Generic (PLEG): container finished" podID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerID="0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea" exitCode=0 Sep 17 17:46:20.216016 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.215611 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" Sep 17 17:46:20.216016 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.215650 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" event={"ID":"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3","Type":"ContainerDied","Data":"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea"} Sep 17 17:46:20.216016 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.215699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2" event={"ID":"7f5664c0-32ef-4d2a-a242-53ecdb80cfa3","Type":"ContainerDied","Data":"e25d9d446685a42235880c10f8bb9002c0fe63b5c894bddc663fb95635c85d80"} Sep 17 17:46:20.216016 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.215723 2583 scope.go:117] "RemoveContainer" containerID="0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea" Sep 17 17:46:20.224015 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.223997 2583 scope.go:117] "RemoveContainer" containerID="0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea" Sep 17 17:46:20.224295 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:46:20.224274 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea\": container with ID starting with 0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea not found: ID does not exist" containerID="0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea" Sep 17 17:46:20.224344 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.224303 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea"} err="failed to get container status \"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea\": rpc error: code = NotFound desc = could not find container \"0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea\": container with ID starting with 0e7898120798d227585d515097925840e325f652f817d8c846fc49e2f890e4ea not found: ID does not exist" Sep 17 17:46:20.237426 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.237403 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:46:20.241570 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:20.241547 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5b8c1-7f5b78944d-ngml2"] Sep 17 17:46:21.222483 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:21.222453 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" path="/var/lib/kubelet/pods/7f5664c0-32ef-4d2a-a242-53ecdb80cfa3/volumes" Sep 17 17:46:39.242518 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.242483 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:46:39.243027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.242861 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" Sep 17 17:46:39.243027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.242878 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" Sep 17 17:46:39.243027 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.242972 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5664c0-32ef-4d2a-a242-53ecdb80cfa3" containerName="switch-graph-5b8c1" Sep 17 17:46:39.245876 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.245855 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.248683 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.248661 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 17 17:46:39.248780 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.248753 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-28pvs\"" Sep 17 17:46:39.248780 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.248765 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:46:39.249901 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.249887 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 17 17:46:39.258554 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.258533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:46:39.340664 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.340636 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.340807 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.340686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.441645 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.441606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.441783 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.441673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.441783 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:46:39.441750 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Sep 17 17:46:39.441863 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:46:39.441852 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls podName:0b691b26-f64b-4c40-8403-f5e4aec2cfc1 nodeName:}" failed. No retries permitted until 2026-09-17 17:46:39.941830934 +0000 UTC m=+749.187871722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls") pod "model-chainer-7cf9877cc4-6tdsb" (UID: "0b691b26-f64b-4c40-8403-f5e4aec2cfc1") : secret "model-chainer-serving-cert" not found Sep 17 17:46:39.442363 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.442348 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.944793 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.944758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:39.947166 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:39.947137 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") pod \"model-chainer-7cf9877cc4-6tdsb\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:40.155786 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:40.155764 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:40.425869 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:40.425842 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:46:40.428130 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:46:40.428095 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b691b26_f64b_4c40_8403_f5e4aec2cfc1.slice/crio-a6de207c7c2bd1310522bdbef3f2214112f0d426612a653dd2b7a810a31b56c4 WatchSource:0}: Error finding container a6de207c7c2bd1310522bdbef3f2214112f0d426612a653dd2b7a810a31b56c4: Status 404 returned error can't find the container with id a6de207c7c2bd1310522bdbef3f2214112f0d426612a653dd2b7a810a31b56c4 Sep 17 17:46:41.298551 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:41.298514 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" event={"ID":"0b691b26-f64b-4c40-8403-f5e4aec2cfc1","Type":"ContainerStarted","Data":"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02"} Sep 17 17:46:41.298740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:41.298561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" event={"ID":"0b691b26-f64b-4c40-8403-f5e4aec2cfc1","Type":"ContainerStarted","Data":"a6de207c7c2bd1310522bdbef3f2214112f0d426612a653dd2b7a810a31b56c4"} Sep 17 17:46:41.298740 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:41.298635 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:41.316709 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:41.316662 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podStartSLOduration=2.316646761 podStartE2EDuration="2.316646761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:46:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:46:41.315804639 +0000 UTC m=+750.561845455" watchObservedRunningTime="2026-09-17 17:46:41.316646761 +0000 UTC m=+750.562687565" Sep 17 17:46:47.307423 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:47.307394 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:46:49.702571 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.702537 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:46:49.706040 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.706024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.709320 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.709300 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-2d039-serving-cert\"" Sep 17 17:46:49.709434 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.709325 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-2d039-kube-rbac-proxy-sar-config\"" Sep 17 17:46:49.715419 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.715401 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:46:49.811076 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.811056 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.811189 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.811100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.885918 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.885890 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:46:49.886103 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.886080 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" containerID="cri-o://443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02" gracePeriod=30 Sep 17 17:46:49.911711 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.911685 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.911803 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.911744 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.912301 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.912285 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:49.913967 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:49.913952 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls\") pod \"switch-graph-2d039-5d68f9b84-9fcp8\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:50.015744 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:50.015683 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:50.294308 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:50.294275 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:46:50.301329 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:46:50.301304 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58fe8576_f9ed_44ee_85fa_7d1c52a7c157.slice/crio-4f0053824811196c7a7fe9f82bc0be7bb5dca169e3b5a02b9239ef64509b74c1 WatchSource:0}: Error finding container 4f0053824811196c7a7fe9f82bc0be7bb5dca169e3b5a02b9239ef64509b74c1: Status 404 returned error can't find the container with id 4f0053824811196c7a7fe9f82bc0be7bb5dca169e3b5a02b9239ef64509b74c1 Sep 17 17:46:50.331003 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:50.330978 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" event={"ID":"58fe8576-f9ed-44ee-85fa-7d1c52a7c157","Type":"ContainerStarted","Data":"4f0053824811196c7a7fe9f82bc0be7bb5dca169e3b5a02b9239ef64509b74c1"} Sep 17 17:46:51.336036 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:51.336005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" event={"ID":"58fe8576-f9ed-44ee-85fa-7d1c52a7c157","Type":"ContainerStarted","Data":"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9"} Sep 17 17:46:51.336425 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:51.336284 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:46:51.355723 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:51.355679 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podStartSLOduration=2.355662748 podStartE2EDuration="2.355662748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:46:51.353858767 +0000 UTC m=+760.599899585" watchObservedRunningTime="2026-09-17 17:46:51.355662748 +0000 UTC m=+760.601703555" Sep 17 17:46:52.306435 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:52.306398 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:57.306432 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:57.306394 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:46:57.345988 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:46:57.345959 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:47:02.306058 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:02.306019 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:47:02.311144 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:02.307005 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:47:07.306279 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:07.306238 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:47:12.306051 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:12.306012 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:47:17.306490 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:17.306442 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:47:20.034898 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.034872 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:47:20.131473 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.131440 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") pod \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " Sep 17 17:47:20.131608 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.131525 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle\") pod \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\" (UID: \"0b691b26-f64b-4c40-8403-f5e4aec2cfc1\") " Sep 17 17:47:20.131853 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.131828 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0b691b26-f64b-4c40-8403-f5e4aec2cfc1" (UID: "0b691b26-f64b-4c40-8403-f5e4aec2cfc1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:47:20.133455 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.133432 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0b691b26-f64b-4c40-8403-f5e4aec2cfc1" (UID: "0b691b26-f64b-4c40-8403-f5e4aec2cfc1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:47:20.232053 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.231995 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:47:20.232053 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.232026 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b691b26-f64b-4c40-8403-f5e4aec2cfc1-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:47:20.445757 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.445727 2583 generic.go:356] "Generic (PLEG): container finished" podID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerID="443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02" exitCode=0 Sep 17 17:47:20.445877 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.445775 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" Sep 17 17:47:20.445877 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.445788 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" event={"ID":"0b691b26-f64b-4c40-8403-f5e4aec2cfc1","Type":"ContainerDied","Data":"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02"} Sep 17 17:47:20.445877 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.445839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb" event={"ID":"0b691b26-f64b-4c40-8403-f5e4aec2cfc1","Type":"ContainerDied","Data":"a6de207c7c2bd1310522bdbef3f2214112f0d426612a653dd2b7a810a31b56c4"} Sep 17 17:47:20.445877 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.445868 2583 scope.go:117] "RemoveContainer" containerID="443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02" Sep 17 17:47:20.455144 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.455108 2583 scope.go:117] "RemoveContainer" containerID="443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02" Sep 17 17:47:20.455411 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:47:20.455390 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02\": container with ID starting with 443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02 not found: ID does not exist" containerID="443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02" Sep 17 17:47:20.455491 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.455419 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02"} err="failed to get container status \"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02\": rpc error: code = NotFound desc = could not find container \"443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02\": container with ID starting with 443cd939b39170e2bd8f9396e0e1a06d41ea5d120307cbfeca12219e6f4abc02 not found: ID does not exist" Sep 17 17:47:20.466879 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.466856 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:47:20.470607 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:20.470587 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7cf9877cc4-6tdsb"] Sep 17 17:47:21.223427 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:21.223392 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" path="/var/lib/kubelet/pods/0b691b26-f64b-4c40-8403-f5e4aec2cfc1/volumes" Sep 17 17:47:50.048505 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.048469 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:47:50.048854 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.048728 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" Sep 17 17:47:50.048854 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.048738 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" Sep 17 17:47:50.048854 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.048801 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b691b26-f64b-4c40-8403-f5e4aec2cfc1" containerName="model-chainer" Sep 17 17:47:50.051480 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.051465 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.054000 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.053979 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-390a8-kube-rbac-proxy-sar-config\"" Sep 17 17:47:50.054103 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.054005 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-390a8-serving-cert\"" Sep 17 17:47:50.060461 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.060437 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:47:50.133736 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.133710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.133837 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.133744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.234896 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.234869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.235017 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.234900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.235017 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:47:50.235006 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-390a8-serving-cert: secret "sequence-graph-390a8-serving-cert" not found Sep 17 17:47:50.235169 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:47:50.235064 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls podName:14121ca3-b64b-446c-aa51-106ab342feaa nodeName:}" failed. No retries permitted until 2026-09-17 17:47:50.735049875 +0000 UTC m=+819.981090657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls") pod "sequence-graph-390a8-857fc7d7c6-qzhnq" (UID: "14121ca3-b64b-446c-aa51-106ab342feaa") : secret "sequence-graph-390a8-serving-cert" not found Sep 17 17:47:50.235572 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.235548 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.738956 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.738929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.741209 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.741181 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") pod \"sequence-graph-390a8-857fc7d7c6-qzhnq\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:50.961313 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:50.961290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:51.231444 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:51.231422 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:47:51.233758 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:47:51.233734 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14121ca3_b64b_446c_aa51_106ab342feaa.slice/crio-fcca90e54eecfb19595b80d70dd285b5e45fd1d3f91b70332fcffc968b99f858 WatchSource:0}: Error finding container fcca90e54eecfb19595b80d70dd285b5e45fd1d3f91b70332fcffc968b99f858: Status 404 returned error can't find the container with id fcca90e54eecfb19595b80d70dd285b5e45fd1d3f91b70332fcffc968b99f858 Sep 17 17:47:51.569532 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:51.569497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" event={"ID":"14121ca3-b64b-446c-aa51-106ab342feaa","Type":"ContainerStarted","Data":"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb"} Sep 17 17:47:51.569673 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:51.569560 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" event={"ID":"14121ca3-b64b-446c-aa51-106ab342feaa","Type":"ContainerStarted","Data":"fcca90e54eecfb19595b80d70dd285b5e45fd1d3f91b70332fcffc968b99f858"} Sep 17 17:47:51.569673 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:51.569610 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:47:51.586186 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:51.586138 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podStartSLOduration=1.586103784 podStartE2EDuration="1.586103784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:47:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:47:51.586028272 +0000 UTC m=+820.832069080" watchObservedRunningTime="2026-09-17 17:47:51.586103784 +0000 UTC m=+820.832144590" Sep 17 17:47:57.579483 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:47:57.579447 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:49:11.146051 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:49:11.145950 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:49:11.149639 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:49:11.149613 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:54:11.178724 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:54:11.178696 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:54:11.183160 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:54:11.183107 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:55:04.148794 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:04.148719 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:55:04.149337 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:04.148909 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" containerID="cri-o://ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9" gracePeriod=30 Sep 17 17:55:07.344935 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:07.344893 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:12.344276 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:12.344229 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:17.344434 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:17.344387 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:17.344940 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:17.344498 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:55:22.344486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:22.344438 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:27.344800 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:27.344755 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:32.344602 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:32.344559 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:55:34.283684 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.283661 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:55:34.310623 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.310602 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle\") pod \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " Sep 17 17:55:34.310741 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.310630 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls\") pod \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\" (UID: \"58fe8576-f9ed-44ee-85fa-7d1c52a7c157\") " Sep 17 17:55:34.310939 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.310921 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "58fe8576-f9ed-44ee-85fa-7d1c52a7c157" (UID: "58fe8576-f9ed-44ee-85fa-7d1c52a7c157"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:55:34.312520 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.312495 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "58fe8576-f9ed-44ee-85fa-7d1c52a7c157" (UID: "58fe8576-f9ed-44ee-85fa-7d1c52a7c157"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:55:34.411352 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.411302 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:55:34.411352 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.411325 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58fe8576-f9ed-44ee-85fa-7d1c52a7c157-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:55:34.430514 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.430470 2583 generic.go:356] "Generic (PLEG): container finished" podID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerID="ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9" exitCode=0 Sep 17 17:55:34.430617 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.430526 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" Sep 17 17:55:34.430617 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.430548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" event={"ID":"58fe8576-f9ed-44ee-85fa-7d1c52a7c157","Type":"ContainerDied","Data":"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9"} Sep 17 17:55:34.430617 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.430599 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8" event={"ID":"58fe8576-f9ed-44ee-85fa-7d1c52a7c157","Type":"ContainerDied","Data":"4f0053824811196c7a7fe9f82bc0be7bb5dca169e3b5a02b9239ef64509b74c1"} Sep 17 17:55:34.430772 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.430622 2583 scope.go:117] "RemoveContainer" containerID="ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9" Sep 17 17:55:34.439254 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.439233 2583 scope.go:117] "RemoveContainer" containerID="ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9" Sep 17 17:55:34.439598 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:55:34.439579 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9\": container with ID starting with ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9 not found: ID does not exist" containerID="ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9" Sep 17 17:55:34.439658 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.439606 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9"} err="failed to get container status \"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9\": rpc error: code = NotFound desc = could not find container \"ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9\": container with ID starting with ec2b92aa906483f445c0e91655cc54b17034a1c304f746b92aa9340cb99787c9 not found: ID does not exist" Sep 17 17:55:34.453077 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.453047 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:55:34.456934 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:34.456913 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-2d039-5d68f9b84-9fcp8"] Sep 17 17:55:35.223878 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:55:35.223841 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" path="/var/lib/kubelet/pods/58fe8576-f9ed-44ee-85fa-7d1c52a7c157/volumes" Sep 17 17:56:04.410075 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.410040 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:04.410565 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.410356 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" Sep 17 17:56:04.410565 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.410370 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" Sep 17 17:56:04.410565 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.410458 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58fe8576-f9ed-44ee-85fa-7d1c52a7c157" containerName="switch-graph-2d039" Sep 17 17:56:04.413628 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.413609 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.416085 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.416058 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2b6c1-kube-rbac-proxy-sar-config\"" Sep 17 17:56:04.416278 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.416064 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2b6c1-serving-cert\"" Sep 17 17:56:04.426703 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.426667 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:04.523925 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.523896 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.524058 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.523966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.624312 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.624285 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.624423 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.624325 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.624857 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.624838 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.626648 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.626629 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls\") pod \"ensemble-graph-2b6c1-587666d7dc-k2xzd\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.729677 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.729617 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:04.772152 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.772105 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:56:04.772417 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:04.772371 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" containerID="cri-o://00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb" gracePeriod=30 Sep 17 17:56:05.003732 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:05.003640 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:05.006305 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:56:05.006274 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f336a7_8972_495d_87df_886d7f5b74b4.slice/crio-abe1321c76b1167cd1ebf0736658bd04e13775c42403c74dfac5c3440575277c WatchSource:0}: Error finding container abe1321c76b1167cd1ebf0736658bd04e13775c42403c74dfac5c3440575277c: Status 404 returned error can't find the container with id abe1321c76b1167cd1ebf0736658bd04e13775c42403c74dfac5c3440575277c Sep 17 17:56:05.551712 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:05.551674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" event={"ID":"30f336a7-8972-495d-87df-886d7f5b74b4","Type":"ContainerStarted","Data":"955b33f385dc99bb143f25f7aa6d6c9c8c061e1e95832b59947f1b4609b2e395"} Sep 17 17:56:05.552092 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:05.551737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" event={"ID":"30f336a7-8972-495d-87df-886d7f5b74b4","Type":"ContainerStarted","Data":"abe1321c76b1167cd1ebf0736658bd04e13775c42403c74dfac5c3440575277c"} Sep 17 17:56:05.552092 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:05.551767 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:05.574597 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:05.574547 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podStartSLOduration=1.574529549 podStartE2EDuration="1.574529549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:05.573777078 +0000 UTC m=+1314.819817883" watchObservedRunningTime="2026-09-17 17:56:05.574529549 +0000 UTC m=+1314.820570352" Sep 17 17:56:07.578266 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:07.578219 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:11.560972 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:11.560944 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:12.578315 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:12.578261 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:17.577869 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:17.577827 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:17.578433 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:17.577962 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:56:18.470316 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:18.470274 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:18.470511 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:18.470471 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" containerID="cri-o://955b33f385dc99bb143f25f7aa6d6c9c8c061e1e95832b59947f1b4609b2e395" gracePeriod=30 Sep 17 17:56:21.560024 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:21.559978 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:22.577649 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:22.577605 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:26.560053 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:26.560006 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:27.577708 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:27.577665 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:31.559956 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:31.559849 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:31.560386 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:31.559977 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:32.578072 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:32.578038 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:34.922238 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:34.922215 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:56:35.029341 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.029261 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") pod \"14121ca3-b64b-446c-aa51-106ab342feaa\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " Sep 17 17:56:35.029341 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.029325 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle\") pod \"14121ca3-b64b-446c-aa51-106ab342feaa\" (UID: \"14121ca3-b64b-446c-aa51-106ab342feaa\") " Sep 17 17:56:35.029680 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.029654 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "14121ca3-b64b-446c-aa51-106ab342feaa" (UID: "14121ca3-b64b-446c-aa51-106ab342feaa"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:56:35.031176 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.031150 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "14121ca3-b64b-446c-aa51-106ab342feaa" (UID: "14121ca3-b64b-446c-aa51-106ab342feaa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:56:35.130251 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.130232 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14121ca3-b64b-446c-aa51-106ab342feaa-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:56:35.130251 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.130250 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14121ca3-b64b-446c-aa51-106ab342feaa-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:56:35.674753 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.674722 2583 generic.go:356] "Generic (PLEG): container finished" podID="14121ca3-b64b-446c-aa51-106ab342feaa" containerID="00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb" exitCode=0 Sep 17 17:56:35.674863 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.674761 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" event={"ID":"14121ca3-b64b-446c-aa51-106ab342feaa","Type":"ContainerDied","Data":"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb"} Sep 17 17:56:35.674863 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.674770 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" Sep 17 17:56:35.674863 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.674803 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq" event={"ID":"14121ca3-b64b-446c-aa51-106ab342feaa","Type":"ContainerDied","Data":"fcca90e54eecfb19595b80d70dd285b5e45fd1d3f91b70332fcffc968b99f858"} Sep 17 17:56:35.674863 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.674829 2583 scope.go:117] "RemoveContainer" containerID="00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb" Sep 17 17:56:35.683672 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.683653 2583 scope.go:117] "RemoveContainer" containerID="00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb" Sep 17 17:56:35.684002 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:56:35.683973 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb\": container with ID starting with 00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb not found: ID does not exist" containerID="00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb" Sep 17 17:56:35.684077 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.684014 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb"} err="failed to get container status \"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb\": rpc error: code = NotFound desc = could not find container \"00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb\": container with ID starting with 00590d4805d51b4116df71ddada408039e9c57747bb7db444133568635f30acb not found: ID does not exist" Sep 17 17:56:35.699029 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.698998 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:56:35.703636 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:35.703606 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-390a8-857fc7d7c6-qzhnq"] Sep 17 17:56:36.559989 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:36.559951 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:37.222620 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:37.222591 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" path="/var/lib/kubelet/pods/14121ca3-b64b-446c-aa51-106ab342feaa/volumes" Sep 17 17:56:41.559642 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:41.559606 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:46.559333 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:46.559285 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:48.730455 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:48.730383 2583 generic.go:356] "Generic (PLEG): container finished" podID="30f336a7-8972-495d-87df-886d7f5b74b4" containerID="955b33f385dc99bb143f25f7aa6d6c9c8c061e1e95832b59947f1b4609b2e395" exitCode=0 Sep 17 17:56:48.730768 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:48.730459 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" event={"ID":"30f336a7-8972-495d-87df-886d7f5b74b4","Type":"ContainerDied","Data":"955b33f385dc99bb143f25f7aa6d6c9c8c061e1e95832b59947f1b4609b2e395"} Sep 17 17:56:49.110166 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.110142 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:49.228501 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.228478 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls\") pod \"30f336a7-8972-495d-87df-886d7f5b74b4\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " Sep 17 17:56:49.228600 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.228526 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle\") pod \"30f336a7-8972-495d-87df-886d7f5b74b4\" (UID: \"30f336a7-8972-495d-87df-886d7f5b74b4\") " Sep 17 17:56:49.228865 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.228840 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "30f336a7-8972-495d-87df-886d7f5b74b4" (UID: "30f336a7-8972-495d-87df-886d7f5b74b4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:56:49.230589 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.230565 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "30f336a7-8972-495d-87df-886d7f5b74b4" (UID: "30f336a7-8972-495d-87df-886d7f5b74b4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:56:49.329284 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.329243 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30f336a7-8972-495d-87df-886d7f5b74b4-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:56:49.329373 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.329285 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30f336a7-8972-495d-87df-886d7f5b74b4-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:56:49.736125 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.736049 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" Sep 17 17:56:49.736125 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.736062 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd" event={"ID":"30f336a7-8972-495d-87df-886d7f5b74b4","Type":"ContainerDied","Data":"abe1321c76b1167cd1ebf0736658bd04e13775c42403c74dfac5c3440575277c"} Sep 17 17:56:49.736557 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.736142 2583 scope.go:117] "RemoveContainer" containerID="955b33f385dc99bb143f25f7aa6d6c9c8c061e1e95832b59947f1b4609b2e395" Sep 17 17:56:49.758206 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.758179 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:49.763671 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:49.763645 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2b6c1-587666d7dc-k2xzd"] Sep 17 17:56:51.222633 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:56:51.222591 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" path="/var/lib/kubelet/pods/30f336a7-8972-495d-87df-886d7f5b74b4/volumes" Sep 17 17:57:14.997134 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997080 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997460 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997478 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997498 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997505 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997570 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f336a7-8972-495d-87df-886d7f5b74b4" containerName="ensemble-graph-2b6c1" Sep 17 17:57:14.997650 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:14.997583 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="14121ca3-b64b-446c-aa51-106ab342feaa" containerName="sequence-graph-390a8" Sep 17 17:57:15.001974 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.001952 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.004388 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.004363 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-6b6c1-serving-cert\"" Sep 17 17:57:15.004553 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.004532 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-6b6c1-kube-rbac-proxy-sar-config\"" Sep 17 17:57:15.004623 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.004589 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-28pvs\"" Sep 17 17:57:15.005386 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.005367 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:57:15.011226 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.011203 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:57:15.106131 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.106092 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.106235 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.106143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.207073 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.207047 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.207192 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.207101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.207240 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:57:15.207210 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-6b6c1-serving-cert: secret "sequence-graph-6b6c1-serving-cert" not found Sep 17 17:57:15.207282 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:57:15.207271 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls podName:d23eee3a-d98a-4848-9877-9f051241ce51 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:15.707255608 +0000 UTC m=+1384.953296391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls") pod "sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" (UID: "d23eee3a-d98a-4848-9877-9f051241ce51") : secret "sequence-graph-6b6c1-serving-cert" not found Sep 17 17:57:15.207647 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.207632 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.711486 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.711448 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.714040 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.714008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") pod \"sequence-graph-6b6c1-7cb7fbb4f9-xm9z6\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:15.912587 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:15.912563 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:16.187933 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:16.187911 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:57:16.189969 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:57:16.189943 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23eee3a_d98a_4848_9877_9f051241ce51.slice/crio-deebbae700e607b5e5b60f50a70af9f2cc9a8cd07b63df9b544ea7a9363bbe4d WatchSource:0}: Error finding container deebbae700e607b5e5b60f50a70af9f2cc9a8cd07b63df9b544ea7a9363bbe4d: Status 404 returned error can't find the container with id deebbae700e607b5e5b60f50a70af9f2cc9a8cd07b63df9b544ea7a9363bbe4d Sep 17 17:57:16.854827 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:16.854791 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" event={"ID":"d23eee3a-d98a-4848-9877-9f051241ce51","Type":"ContainerStarted","Data":"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22"} Sep 17 17:57:16.855020 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:16.854851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" event={"ID":"d23eee3a-d98a-4848-9877-9f051241ce51","Type":"ContainerStarted","Data":"deebbae700e607b5e5b60f50a70af9f2cc9a8cd07b63df9b544ea7a9363bbe4d"} Sep 17 17:57:16.855020 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:16.854874 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:16.873655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:16.873605 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podStartSLOduration=2.8735881 podStartE2EDuration="2.8735881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:16.871505406 +0000 UTC m=+1386.117546210" watchObservedRunningTime="2026-09-17 17:57:16.8735881 +0000 UTC m=+1386.119628919" Sep 17 17:57:18.730448 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.730417 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 17:57:18.733452 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.733433 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:18.735722 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.735702 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c5c33-kube-rbac-proxy-sar-config\"" Sep 17 17:57:18.735825 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.735706 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c5c33-serving-cert\"" Sep 17 17:57:18.742622 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.742597 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 17:57:18.833680 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.833656 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:18.833784 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.833698 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:18.934048 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.934021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:18.934157 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.934075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:18.934215 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:57:18.934202 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-c5c33-serving-cert: secret "ensemble-graph-c5c33-serving-cert" not found Sep 17 17:57:18.934266 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:57:18.934256 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls podName:a5dae2f9-d6ee-4d47-9c29-43d447468fed nodeName:}" failed. No retries permitted until 2026-09-17 17:57:19.434239526 +0000 UTC m=+1388.680280308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls") pod "ensemble-graph-c5c33-68c85bbb54-slzk5" (UID: "a5dae2f9-d6ee-4d47-9c29-43d447468fed") : secret "ensemble-graph-c5c33-serving-cert" not found Sep 17 17:57:18.934620 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:18.934601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:19.438530 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:19.438503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:19.441134 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:19.441083 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") pod \"ensemble-graph-c5c33-68c85bbb54-slzk5\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:19.645374 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:19.645346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:19.917806 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:19.917785 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 17:57:19.919805 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:57:19.919779 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5dae2f9_d6ee_4d47_9c29_43d447468fed.slice/crio-8074262b39b897fef66d5164fe81529a0f2b6f5b25e5376e8261deabe0d98a37 WatchSource:0}: Error finding container 8074262b39b897fef66d5164fe81529a0f2b6f5b25e5376e8261deabe0d98a37: Status 404 returned error can't find the container with id 8074262b39b897fef66d5164fe81529a0f2b6f5b25e5376e8261deabe0d98a37 Sep 17 17:57:20.872346 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:20.872305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" event={"ID":"a5dae2f9-d6ee-4d47-9c29-43d447468fed","Type":"ContainerStarted","Data":"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e"} Sep 17 17:57:20.872577 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:20.872490 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:20.872577 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:20.872526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" event={"ID":"a5dae2f9-d6ee-4d47-9c29-43d447468fed","Type":"ContainerStarted","Data":"8074262b39b897fef66d5164fe81529a0f2b6f5b25e5376e8261deabe0d98a37"} Sep 17 17:57:20.891348 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:20.891297 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podStartSLOduration=2.891281256 podStartE2EDuration="2.891281256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:20.890317981 +0000 UTC m=+1390.136358790" watchObservedRunningTime="2026-09-17 17:57:20.891281256 +0000 UTC m=+1390.137322059" Sep 17 17:57:22.865180 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:22.865146 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:26.882432 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:26.882391 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 17:57:29.086985 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:29.086949 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:57:29.087396 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:29.087202 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" containerID="cri-o://419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22" gracePeriod=30 Sep 17 17:57:32.862543 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:32.862508 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:37.862647 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:37.862604 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:42.862378 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:42.862333 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:42.862787 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:42.862449 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:47.863028 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:47.862982 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:52.862678 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:52.862634 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:57.862776 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:57.862729 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:59.229069 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.229048 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:57:59.305341 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.305317 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") pod \"d23eee3a-d98a-4848-9877-9f051241ce51\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " Sep 17 17:57:59.305447 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.305354 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle\") pod \"d23eee3a-d98a-4848-9877-9f051241ce51\" (UID: \"d23eee3a-d98a-4848-9877-9f051241ce51\") " Sep 17 17:57:59.305727 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.305704 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d23eee3a-d98a-4848-9877-9f051241ce51" (UID: "d23eee3a-d98a-4848-9877-9f051241ce51"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:57:59.307145 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.307110 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d23eee3a-d98a-4848-9877-9f051241ce51" (UID: "d23eee3a-d98a-4848-9877-9f051241ce51"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:57:59.406473 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.406419 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d23eee3a-d98a-4848-9877-9f051241ce51-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:57:59.406473 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:57:59.406442 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d23eee3a-d98a-4848-9877-9f051241ce51-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 17:58:00.032412 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.032381 2583 generic.go:356] "Generic (PLEG): container finished" podID="d23eee3a-d98a-4848-9877-9f051241ce51" containerID="419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22" exitCode=0 Sep 17 17:58:00.032552 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.032431 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" Sep 17 17:58:00.032552 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.032459 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" event={"ID":"d23eee3a-d98a-4848-9877-9f051241ce51","Type":"ContainerDied","Data":"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22"} Sep 17 17:58:00.032552 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.032520 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6" event={"ID":"d23eee3a-d98a-4848-9877-9f051241ce51","Type":"ContainerDied","Data":"deebbae700e607b5e5b60f50a70af9f2cc9a8cd07b63df9b544ea7a9363bbe4d"} Sep 17 17:58:00.032552 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.032544 2583 scope.go:117] "RemoveContainer" containerID="419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22" Sep 17 17:58:00.042086 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.042067 2583 scope.go:117] "RemoveContainer" containerID="419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22" Sep 17 17:58:00.042439 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:58:00.042420 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22\": container with ID starting with 419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22 not found: ID does not exist" containerID="419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22" Sep 17 17:58:00.042528 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.042451 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22"} err="failed to get container status \"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22\": rpc error: code = NotFound desc = could not find container \"419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22\": container with ID starting with 419751c50ffe2e583bdd407a7c3c40d952b8a74be300d0e28e32e4640911cb22 not found: ID does not exist" Sep 17 17:58:00.056535 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.056510 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:58:00.065452 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:00.065429 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6b6c1-7cb7fbb4f9-xm9z6"] Sep 17 17:58:01.223515 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:01.223484 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" path="/var/lib/kubelet/pods/d23eee3a-d98a-4848-9877-9f051241ce51/volumes" Sep 17 17:58:29.347690 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.347660 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 17:58:29.348064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.347925 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" Sep 17 17:58:29.348064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.347935 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" Sep 17 17:58:29.348064 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.347993 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23eee3a-d98a-4848-9877-9f051241ce51" containerName="sequence-graph-6b6c1" Sep 17 17:58:29.352158 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.352139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:29.354743 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.354717 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f13ae-serving-cert\"" Sep 17 17:58:29.354743 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.354722 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f13ae-kube-rbac-proxy-sar-config\"" Sep 17 17:58:29.368317 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.366512 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 17:58:29.411724 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.411700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:29.411817 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.411744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:29.512655 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.512628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:29.512766 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.512691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:29.512821 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:58:29.512806 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-f13ae-serving-cert: secret "sequence-graph-f13ae-serving-cert" not found Sep 17 17:58:29.512889 ip-10-0-128-248 kubenswrapper[2583]: E0917 17:58:29.512879 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls podName:50e84966-d1db-4ed5-92b5-0e3f25094d75 nodeName:}" failed. No retries permitted until 2026-09-17 17:58:30.012856684 +0000 UTC m=+1459.258897482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls") pod "sequence-graph-f13ae-d54cbf9fb-xkjfq" (UID: "50e84966-d1db-4ed5-92b5-0e3f25094d75") : secret "sequence-graph-f13ae-serving-cert" not found Sep 17 17:58:29.513375 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:29.513357 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:30.016687 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:30.016657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:30.019321 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:30.019298 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") pod \"sequence-graph-f13ae-d54cbf9fb-xkjfq\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:30.261632 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:30.261604 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:30.535599 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:30.535571 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 17:58:30.538151 ip-10-0-128-248 kubenswrapper[2583]: W0917 17:58:30.538102 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e84966_d1db_4ed5_92b5_0e3f25094d75.slice/crio-68133dd8a3b218114c4c28cd8cb2c7ce91414414506136da504fbd62faab7763 WatchSource:0}: Error finding container 68133dd8a3b218114c4c28cd8cb2c7ce91414414506136da504fbd62faab7763: Status 404 returned error can't find the container with id 68133dd8a3b218114c4c28cd8cb2c7ce91414414506136da504fbd62faab7763 Sep 17 17:58:31.168148 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:31.168097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" event={"ID":"50e84966-d1db-4ed5-92b5-0e3f25094d75","Type":"ContainerStarted","Data":"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045"} Sep 17 17:58:31.168299 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:31.168174 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" event={"ID":"50e84966-d1db-4ed5-92b5-0e3f25094d75","Type":"ContainerStarted","Data":"68133dd8a3b218114c4c28cd8cb2c7ce91414414506136da504fbd62faab7763"} Sep 17 17:58:31.168299 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:31.168255 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:58:31.187750 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:31.187700 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podStartSLOduration=2.187683212 podStartE2EDuration="2.187683212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:58:31.186978698 +0000 UTC m=+1460.433019502" watchObservedRunningTime="2026-09-17 17:58:31.187683212 +0000 UTC m=+1460.433724015" Sep 17 17:58:37.177609 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:58:37.177572 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 17:59:11.205771 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:59:11.205741 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 17:59:11.211524 ip-10-0-128-248 kubenswrapper[2583]: I0917 17:59:11.211499 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:04:11.233418 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:04:11.233386 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:04:11.239501 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:04:11.239477 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:05:33.322755 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:33.322681 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 18:05:33.323308 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:33.322892 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" containerID="cri-o://c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e" gracePeriod=30 Sep 17 18:05:36.881100 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:36.881057 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:41.881515 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:41.881470 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:46.880715 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:46.880672 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:46.881108 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:46.880785 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 18:05:51.880951 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:51.880900 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:56.881084 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:05:56.881042 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:01.881352 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:01.881304 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:03.474106 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.474084 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 18:06:03.648331 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.648270 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle\") pod \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " Sep 17 18:06:03.648445 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.648348 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") pod \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\" (UID: \"a5dae2f9-d6ee-4d47-9c29-43d447468fed\") " Sep 17 18:06:03.648617 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.648593 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a5dae2f9-d6ee-4d47-9c29-43d447468fed" (UID: "a5dae2f9-d6ee-4d47-9c29-43d447468fed"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:06:03.650363 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.650338 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a5dae2f9-d6ee-4d47-9c29-43d447468fed" (UID: "a5dae2f9-d6ee-4d47-9c29-43d447468fed"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:06:03.749187 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.749153 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5dae2f9-d6ee-4d47-9c29-43d447468fed-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:06:03.749187 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.749186 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a5dae2f9-d6ee-4d47-9c29-43d447468fed-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:06:03.932447 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.932370 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerID="c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e" exitCode=0 Sep 17 18:06:03.932447 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.932425 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" Sep 17 18:06:03.932447 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.932441 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" event={"ID":"a5dae2f9-d6ee-4d47-9c29-43d447468fed","Type":"ContainerDied","Data":"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e"} Sep 17 18:06:03.932632 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.932502 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5" event={"ID":"a5dae2f9-d6ee-4d47-9c29-43d447468fed","Type":"ContainerDied","Data":"8074262b39b897fef66d5164fe81529a0f2b6f5b25e5376e8261deabe0d98a37"} Sep 17 18:06:03.932632 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.932536 2583 scope.go:117] "RemoveContainer" containerID="c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e" Sep 17 18:06:03.942565 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.942538 2583 scope.go:117] "RemoveContainer" containerID="c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e" Sep 17 18:06:03.942862 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:06:03.942842 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e\": container with ID starting with c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e not found: ID does not exist" containerID="c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e" Sep 17 18:06:03.942939 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.942870 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e"} err="failed to get container status \"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e\": rpc error: code = NotFound desc = could not find container \"c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e\": container with ID starting with c73a9813d03e784dbebb817119f78b884d3d024f2958c0c12eb36dab8995fa5e not found: ID does not exist" Sep 17 18:06:03.955719 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.955691 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 18:06:03.960175 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:03.960149 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c5c33-68c85bbb54-slzk5"] Sep 17 18:06:05.223069 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:05.223034 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" path="/var/lib/kubelet/pods/a5dae2f9-d6ee-4d47-9c29-43d447468fed/volumes" Sep 17 18:06:33.594728 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.594697 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:06:33.595178 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.594962 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" Sep 17 18:06:33.595178 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.594972 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" Sep 17 18:06:33.595178 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.595022 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5dae2f9-d6ee-4d47-9c29-43d447468fed" containerName="ensemble-graph-c5c33" Sep 17 18:06:33.598926 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.598909 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.602400 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.602380 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b5897-serving-cert\"" Sep 17 18:06:33.602583 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.602567 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b5897-kube-rbac-proxy-sar-config\"" Sep 17 18:06:33.609941 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.609919 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:06:33.650654 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.650633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.650762 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.650669 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.750942 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.750910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.751109 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.750956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.751590 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.751567 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.753340 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.753320 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls\") pod \"splitter-graph-b5897-5bdb7c67d9-65869\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:33.908870 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:33.908821 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:34.180963 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:34.180935 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:06:34.182045 ip-10-0-128-248 kubenswrapper[2583]: W0917 18:06:34.182015 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dc6a53_ed0d_4407_b9ab_2259ad6d6c04.slice/crio-c2fdfc45364c853813435cd797eac338fc3ec0c92b83a66996b265d54da0267f WatchSource:0}: Error finding container c2fdfc45364c853813435cd797eac338fc3ec0c92b83a66996b265d54da0267f: Status 404 returned error can't find the container with id c2fdfc45364c853813435cd797eac338fc3ec0c92b83a66996b265d54da0267f Sep 17 18:06:35.057188 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:35.057154 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" event={"ID":"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04","Type":"ContainerStarted","Data":"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421"} Sep 17 18:06:35.057568 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:35.057207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" event={"ID":"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04","Type":"ContainerStarted","Data":"c2fdfc45364c853813435cd797eac338fc3ec0c92b83a66996b265d54da0267f"} Sep 17 18:06:35.057568 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:35.057236 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:35.075085 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:35.075040 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podStartSLOduration=2.075025023 podStartE2EDuration="2.075025023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:35.074023558 +0000 UTC m=+1944.320064362" watchObservedRunningTime="2026-09-17 18:06:35.075025023 +0000 UTC m=+1944.321065826" Sep 17 18:06:41.067193 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:41.067158 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:43.676146 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:43.676058 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:06:43.676526 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:43.676236 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" containerID="cri-o://49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421" gracePeriod=30 Sep 17 18:06:43.976401 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:43.976332 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 18:06:43.976529 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:43.976514 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" containerID="cri-o://60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045" gracePeriod=30 Sep 17 18:06:46.065915 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:46.065873 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:47.175736 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:47.175699 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:51.065518 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:51.065482 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:52.175678 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:52.175634 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:56.065928 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:56.065882 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:56.066365 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:56.066006 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:06:57.175302 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:57.175261 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:57.175702 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:06:57.175374 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 18:07:01.066084 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:01.065991 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:02.175912 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:02.175870 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:06.065500 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:06.065461 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:07.176130 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:07.176070 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:11.065594 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:11.065555 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:12.175943 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:12.175901 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:13.817222 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:13.817183 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:07:13.926155 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:13.926103 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle\") pod \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " Sep 17 18:07:13.926288 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:13.926184 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls\") pod \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\" (UID: \"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04\") " Sep 17 18:07:13.926467 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:13.926443 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" (UID: "72dc6a53-ed0d-4407-b9ab-2259ad6d6c04"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:07:13.928002 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:13.927983 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" (UID: "72dc6a53-ed0d-4407-b9ab-2259ad6d6c04"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:07:14.027151 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.027102 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:07:14.027151 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.027149 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:07:14.102957 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.102936 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 18:07:14.217183 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.217106 2583 generic.go:356] "Generic (PLEG): container finished" podID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerID="60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045" exitCode=0 Sep 17 18:07:14.217183 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.217152 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" event={"ID":"50e84966-d1db-4ed5-92b5-0e3f25094d75","Type":"ContainerDied","Data":"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045"} Sep 17 18:07:14.217183 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.217169 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" Sep 17 18:07:14.217402 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.217207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq" event={"ID":"50e84966-d1db-4ed5-92b5-0e3f25094d75","Type":"ContainerDied","Data":"68133dd8a3b218114c4c28cd8cb2c7ce91414414506136da504fbd62faab7763"} Sep 17 18:07:14.217402 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.217244 2583 scope.go:117] "RemoveContainer" containerID="60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045" Sep 17 18:07:14.218561 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.218542 2583 generic.go:356] "Generic (PLEG): container finished" podID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerID="49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421" exitCode=0 Sep 17 18:07:14.218658 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.218582 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" event={"ID":"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04","Type":"ContainerDied","Data":"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421"} Sep 17 18:07:14.218658 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.218585 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" Sep 17 18:07:14.218658 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.218611 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869" event={"ID":"72dc6a53-ed0d-4407-b9ab-2259ad6d6c04","Type":"ContainerDied","Data":"c2fdfc45364c853813435cd797eac338fc3ec0c92b83a66996b265d54da0267f"} Sep 17 18:07:14.225438 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.225423 2583 scope.go:117] "RemoveContainer" containerID="60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045" Sep 17 18:07:14.225732 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:07:14.225714 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045\": container with ID starting with 60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045 not found: ID does not exist" containerID="60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045" Sep 17 18:07:14.225808 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.225739 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045"} err="failed to get container status \"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045\": rpc error: code = NotFound desc = could not find container \"60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045\": container with ID starting with 60390606b008a99fbeaaca40eaf917dbaab7cdabf53d16c2367f2a7f7d725045 not found: ID does not exist" Sep 17 18:07:14.225808 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.225755 2583 scope.go:117] "RemoveContainer" containerID="49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421" Sep 17 18:07:14.228965 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.228948 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle\") pod \"50e84966-d1db-4ed5-92b5-0e3f25094d75\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " Sep 17 18:07:14.229048 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.228990 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") pod \"50e84966-d1db-4ed5-92b5-0e3f25094d75\" (UID: \"50e84966-d1db-4ed5-92b5-0e3f25094d75\") " Sep 17 18:07:14.229343 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.229319 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "50e84966-d1db-4ed5-92b5-0e3f25094d75" (UID: "50e84966-d1db-4ed5-92b5-0e3f25094d75"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:07:14.230989 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.230966 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50e84966-d1db-4ed5-92b5-0e3f25094d75" (UID: "50e84966-d1db-4ed5-92b5-0e3f25094d75"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:07:14.232852 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.232835 2583 scope.go:117] "RemoveContainer" containerID="49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421" Sep 17 18:07:14.233101 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:07:14.233081 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421\": container with ID starting with 49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421 not found: ID does not exist" containerID="49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421" Sep 17 18:07:14.233165 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.233106 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421"} err="failed to get container status \"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421\": rpc error: code = NotFound desc = could not find container \"49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421\": container with ID starting with 49a84bf862f888cb72e8da507661055459905ecc22660797aede125d9f1ac421 not found: ID does not exist" Sep 17 18:07:14.242884 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.242864 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:07:14.245354 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.245335 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5897-5bdb7c67d9-65869"] Sep 17 18:07:14.330294 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.330274 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50e84966-d1db-4ed5-92b5-0e3f25094d75-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:07:14.330294 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.330294 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50e84966-d1db-4ed5-92b5-0e3f25094d75-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:07:14.539987 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.539964 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 18:07:14.547054 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:14.547027 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f13ae-d54cbf9fb-xkjfq"] Sep 17 18:07:15.223904 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:15.223872 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" path="/var/lib/kubelet/pods/50e84966-d1db-4ed5-92b5-0e3f25094d75/volumes" Sep 17 18:07:15.224332 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:15.224278 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" path="/var/lib/kubelet/pods/72dc6a53-ed0d-4407-b9ab-2259ad6d6c04/volumes" Sep 17 18:07:43.932484 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932446 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932735 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932747 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932758 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932764 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932820 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e84966-d1db-4ed5-92b5-0e3f25094d75" containerName="sequence-graph-f13ae" Sep 17 18:07:43.932876 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.932828 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dc6a53-ed0d-4407-b9ab-2259ad6d6c04" containerName="splitter-graph-b5897" Sep 17 18:07:43.935559 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.935543 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:43.938357 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.938332 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-3ef6b-kube-rbac-proxy-sar-config\"" Sep 17 18:07:43.938357 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.938349 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 18:07:43.938549 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.938330 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-3ef6b-serving-cert\"" Sep 17 18:07:43.938549 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.938338 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-28pvs\"" Sep 17 18:07:43.944830 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:43.944791 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:07:44.033645 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.033619 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.033780 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.033664 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.134969 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.134936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.135079 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.135000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.135581 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.135564 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.137256 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.137236 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls\") pod \"splitter-graph-3ef6b-6679565d6c-hfv7z\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.246253 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.246199 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:44.518697 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:44.518669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:07:44.521375 ip-10-0-128-248 kubenswrapper[2583]: W0917 18:07:44.521342 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23560048_9ee0_4665_a35a_d3feee8ad2e4.slice/crio-b9d2c013584a2395ee6dcc1ab9b2d4e2ccc87132ba8cb9478fd7e2efd0d8d7fe WatchSource:0}: Error finding container b9d2c013584a2395ee6dcc1ab9b2d4e2ccc87132ba8cb9478fd7e2efd0d8d7fe: Status 404 returned error can't find the container with id b9d2c013584a2395ee6dcc1ab9b2d4e2ccc87132ba8cb9478fd7e2efd0d8d7fe Sep 17 18:07:45.338059 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:45.338023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" event={"ID":"23560048-9ee0-4665-a35a-d3feee8ad2e4","Type":"ContainerStarted","Data":"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f"} Sep 17 18:07:45.338544 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:45.338089 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" event={"ID":"23560048-9ee0-4665-a35a-d3feee8ad2e4","Type":"ContainerStarted","Data":"b9d2c013584a2395ee6dcc1ab9b2d4e2ccc87132ba8cb9478fd7e2efd0d8d7fe"} Sep 17 18:07:45.338544 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:45.338151 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:45.356639 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:45.356593 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podStartSLOduration=2.356578789 podStartE2EDuration="2.356578789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:45.355862648 +0000 UTC m=+2014.601903453" watchObservedRunningTime="2026-09-17 18:07:45.356578789 +0000 UTC m=+2014.602619593" Sep 17 18:07:51.348743 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:51.348708 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:07:54.208478 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.208446 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:07:54.211920 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.211900 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.214571 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.214547 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-de145-kube-rbac-proxy-sar-config\"" Sep 17 18:07:54.214665 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.214569 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-de145-serving-cert\"" Sep 17 18:07:54.222130 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.222082 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:07:54.301312 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.301289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.301403 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.301337 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.401702 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.401680 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.401788 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.401731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.401841 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:07:54.401823 2583 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-de145-serving-cert: secret "switch-graph-de145-serving-cert" not found Sep 17 18:07:54.401889 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:07:54.401880 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls podName:cc8af9b7-3d17-4f23-976e-606bdd1d6890 nodeName:}" failed. No retries permitted until 2026-09-17 18:07:54.901865781 +0000 UTC m=+2024.147906565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls") pod "switch-graph-de145-59f9568d97-5nvf7" (UID: "cc8af9b7-3d17-4f23-976e-606bdd1d6890") : secret "switch-graph-de145-serving-cert" not found Sep 17 18:07:54.402284 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.402268 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.904807 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.904783 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:54.907425 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:54.907397 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") pod \"switch-graph-de145-59f9568d97-5nvf7\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:55.129010 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:55.128973 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:55.407463 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:55.407435 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:07:55.410293 ip-10-0-128-248 kubenswrapper[2583]: W0917 18:07:55.410260 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8af9b7_3d17_4f23_976e_606bdd1d6890.slice/crio-e9cd80eb8b3c8e59a0ea881729c53e3f828dda674c2b3d12f632c2634701ae64 WatchSource:0}: Error finding container e9cd80eb8b3c8e59a0ea881729c53e3f828dda674c2b3d12f632c2634701ae64: Status 404 returned error can't find the container with id e9cd80eb8b3c8e59a0ea881729c53e3f828dda674c2b3d12f632c2634701ae64 Sep 17 18:07:56.383733 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:56.383699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" event={"ID":"cc8af9b7-3d17-4f23-976e-606bdd1d6890","Type":"ContainerStarted","Data":"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d"} Sep 17 18:07:56.383922 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:56.383753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" event={"ID":"cc8af9b7-3d17-4f23-976e-606bdd1d6890","Type":"ContainerStarted","Data":"e9cd80eb8b3c8e59a0ea881729c53e3f828dda674c2b3d12f632c2634701ae64"} Sep 17 18:07:56.383922 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:56.383842 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:07:56.402608 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:07:56.402558 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podStartSLOduration=2.402540965 podStartE2EDuration="2.402540965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:56.401891733 +0000 UTC m=+2025.647932538" watchObservedRunningTime="2026-09-17 18:07:56.402540965 +0000 UTC m=+2025.648581768" Sep 17 18:08:02.393602 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:08:02.393570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:09:11.258597 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:09:11.258570 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:09:11.264522 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:09:11.264499 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:14:11.282407 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:14:11.282378 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:14:11.288157 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:14:11.288137 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:15:58.537559 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:15:58.537525 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:15:58.538068 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:15:58.537743 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" containerID="cri-o://f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f" gracePeriod=30 Sep 17 18:16:01.347555 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:01.347473 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:06.346705 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:06.346668 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:11.347101 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:11.347056 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:11.347498 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:11.347199 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:16:16.347645 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:16.347600 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:21.347363 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:21.347327 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:26.347448 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:26.347407 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:16:28.681615 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.681588 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:16:28.767690 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.767647 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls\") pod \"23560048-9ee0-4665-a35a-d3feee8ad2e4\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " Sep 17 18:16:28.767890 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.767712 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle\") pod \"23560048-9ee0-4665-a35a-d3feee8ad2e4\" (UID: \"23560048-9ee0-4665-a35a-d3feee8ad2e4\") " Sep 17 18:16:28.768057 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.768032 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "23560048-9ee0-4665-a35a-d3feee8ad2e4" (UID: "23560048-9ee0-4665-a35a-d3feee8ad2e4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:16:28.769774 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.769752 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "23560048-9ee0-4665-a35a-d3feee8ad2e4" (UID: "23560048-9ee0-4665-a35a-d3feee8ad2e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:16:28.868864 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.868771 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23560048-9ee0-4665-a35a-d3feee8ad2e4-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:16:28.868864 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:28.868811 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23560048-9ee0-4665-a35a-d3feee8ad2e4-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:16:29.412824 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.412785 2583 generic.go:356] "Generic (PLEG): container finished" podID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerID="f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f" exitCode=0 Sep 17 18:16:29.412824 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.412824 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" Sep 17 18:16:29.413035 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.412836 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" event={"ID":"23560048-9ee0-4665-a35a-d3feee8ad2e4","Type":"ContainerDied","Data":"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f"} Sep 17 18:16:29.413035 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.412878 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z" event={"ID":"23560048-9ee0-4665-a35a-d3feee8ad2e4","Type":"ContainerDied","Data":"b9d2c013584a2395ee6dcc1ab9b2d4e2ccc87132ba8cb9478fd7e2efd0d8d7fe"} Sep 17 18:16:29.413035 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.412900 2583 scope.go:117] "RemoveContainer" containerID="f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f" Sep 17 18:16:29.420866 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.420847 2583 scope.go:117] "RemoveContainer" containerID="f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f" Sep 17 18:16:29.421162 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:16:29.421142 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f\": container with ID starting with f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f not found: ID does not exist" containerID="f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f" Sep 17 18:16:29.421231 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.421171 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f"} err="failed to get container status \"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f\": rpc error: code = NotFound desc = could not find container \"f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f\": container with ID starting with f73e2fac063b20e0fe830b55c3fb6d18098c69a9080d5dd5f36c0b075261096f not found: ID does not exist" Sep 17 18:16:29.429391 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.429361 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:16:29.433628 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:29.433607 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3ef6b-6679565d6c-hfv7z"] Sep 17 18:16:31.223636 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:16:31.223600 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" path="/var/lib/kubelet/pods/23560048-9ee0-4665-a35a-d3feee8ad2e4/volumes" Sep 17 18:19:11.305597 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:19:11.305472 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:19:11.311651 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:19:11.311631 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:24:11.329806 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:11.329704 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:24:11.340047 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:11.340025 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:24:13.335086 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:13.335044 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:24:13.335574 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:13.335332 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" containerID="cri-o://3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d" gracePeriod=30 Sep 17 18:24:17.392652 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:17.392611 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:22.392579 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:22.392530 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:27.392283 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:27.392233 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:27.392707 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:27.392362 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:24:29.979958 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:29.979924 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:30.797097 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:30.797071 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:31.602952 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:31.602918 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:32.382833 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:32.382801 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:32.392584 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:32.392548 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:33.177628 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:33.177600 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:33.943495 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:33.943466 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:34.749354 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:34.749319 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:35.536195 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:35.536153 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:36.323379 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:36.323350 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:37.094165 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:37.094138 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:37.392491 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:37.392421 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:37.869818 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:37.869788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:38.639770 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:38.639739 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:39.415031 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:39.414995 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:40.204923 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:40.204882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:41.003080 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:41.003052 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-de145-59f9568d97-5nvf7_cc8af9b7-3d17-4f23-976e-606bdd1d6890/switch-graph-de145/0.log" Sep 17 18:24:42.392456 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:42.392412 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:24:43.480027 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.480004 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:24:43.612982 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.612914 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") pod \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " Sep 17 18:24:43.612982 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.612968 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle\") pod \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\" (UID: \"cc8af9b7-3d17-4f23-976e-606bdd1d6890\") " Sep 17 18:24:43.613358 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.613329 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cc8af9b7-3d17-4f23-976e-606bdd1d6890" (UID: "cc8af9b7-3d17-4f23-976e-606bdd1d6890"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:24:43.615009 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.614987 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cc8af9b7-3d17-4f23-976e-606bdd1d6890" (UID: "cc8af9b7-3d17-4f23-976e-606bdd1d6890"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:24:43.713720 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.713698 2583 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc8af9b7-3d17-4f23-976e-606bdd1d6890-proxy-tls\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:24:43.713720 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:43.713717 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc8af9b7-3d17-4f23-976e-606bdd1d6890-openshift-service-ca-bundle\") on node \"ip-10-0-128-248.ec2.internal\" DevicePath \"\"" Sep 17 18:24:44.316444 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.316412 2583 generic.go:356] "Generic (PLEG): container finished" podID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerID="3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d" exitCode=0 Sep 17 18:24:44.316609 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.316522 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" Sep 17 18:24:44.316609 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.316536 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" event={"ID":"cc8af9b7-3d17-4f23-976e-606bdd1d6890","Type":"ContainerDied","Data":"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d"} Sep 17 18:24:44.316609 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.316590 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7" event={"ID":"cc8af9b7-3d17-4f23-976e-606bdd1d6890","Type":"ContainerDied","Data":"e9cd80eb8b3c8e59a0ea881729c53e3f828dda674c2b3d12f632c2634701ae64"} Sep 17 18:24:44.316779 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.316625 2583 scope.go:117] "RemoveContainer" containerID="3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d" Sep 17 18:24:44.326058 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.326037 2583 scope.go:117] "RemoveContainer" containerID="3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d" Sep 17 18:24:44.326446 ip-10-0-128-248 kubenswrapper[2583]: E0917 18:24:44.326427 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d\": container with ID starting with 3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d not found: ID does not exist" containerID="3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d" Sep 17 18:24:44.326500 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.326456 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d"} err="failed to get container status \"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d\": rpc error: code = NotFound desc = could not find container \"3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d\": container with ID starting with 3c3d0ad48e8246e1b087b6eaf933e8c4b6fe04019efcb40651dcda31c0b0549d not found: ID does not exist" Sep 17 18:24:44.338668 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.338641 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:24:44.344110 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:44.344086 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-de145-59f9568d97-5nvf7"] Sep 17 18:24:45.228684 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:45.228650 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" path="/var/lib/kubelet/pods/cc8af9b7-3d17-4f23-976e-606bdd1d6890/volumes" Sep 17 18:24:48.659823 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:48.659790 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kw8q4_26893c92-4fe2-4746-b1e0-43972e90d6de/global-pull-secret-syncer/0.log" Sep 17 18:24:48.796306 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:48.796277 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wwnvl_95c6bf95-de5b-41a0-b59f-79e4af0016c0/konnectivity-agent/0.log" Sep 17 18:24:48.817094 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:48.817073 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-248.ec2.internal_6df14a0960708fff328f38437ff677d3/haproxy/0.log" Sep 17 18:24:52.717747 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:52.717712 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-kzc24_939b0f33-4382-405b-ab91-17a162e03fbb/cluster-monitoring-operator/0.log" Sep 17 18:24:53.034545 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:53.034520 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jmk6p_e11cfd9c-1afd-4c95-81b0-3fbb07a9d370/node-exporter/0.log" Sep 17 18:24:53.107813 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:53.107792 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jmk6p_e11cfd9c-1afd-4c95-81b0-3fbb07a9d370/kube-rbac-proxy/0.log" Sep 17 18:24:53.190714 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:53.190688 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jmk6p_e11cfd9c-1afd-4c95-81b0-3fbb07a9d370/init-textfile/0.log" Sep 17 18:24:55.236042 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:55.236017 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-mcp9t_d162d8fa-063a-4f39-8d0c-c665350f1ec0/networking-console-plugin/0.log" Sep 17 18:24:55.776646 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:55.776620 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/1.log" Sep 17 18:24:55.781845 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:55.781824 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-4qtmt_f753c88b-c4b9-4666-80b2-48a09074e5e8/console-operator/2.log" Sep 17 18:24:56.764934 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.764905 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-bxjrj_f3c474aa-358d-49fc-b2fd-c8f40d1c6e3d/volume-data-source-validator/0.log" Sep 17 18:24:56.910228 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910199 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4"] Sep 17 18:24:56.910516 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910501 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" Sep 17 18:24:56.910564 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910517 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" Sep 17 18:24:56.910564 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910529 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" Sep 17 18:24:56.910564 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910535 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" Sep 17 18:24:56.910711 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910596 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8af9b7-3d17-4f23-976e-606bdd1d6890" containerName="switch-graph-de145" Sep 17 18:24:56.910711 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.910621 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="23560048-9ee0-4665-a35a-d3feee8ad2e4" containerName="splitter-graph-3ef6b" Sep 17 18:24:56.913902 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.913880 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:56.916558 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.916531 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xfnjl\"/\"default-dockercfg-mwcsm\"" Sep 17 18:24:56.917501 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.917480 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xfnjl\"/\"kube-root-ca.crt\"" Sep 17 18:24:56.917558 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.917482 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xfnjl\"/\"openshift-service-ca.crt\"" Sep 17 18:24:56.926636 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:56.926606 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4"] Sep 17 18:24:57.013460 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.013428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5phvb\" (UniqueName: \"kubernetes.io/projected/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-kube-api-access-5phvb\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.013576 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.013468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-lib-modules\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.013576 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.013486 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-proc\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.013576 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.013545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-sys\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.013698 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.013628 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-podres\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114001 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.113971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5phvb\" (UniqueName: \"kubernetes.io/projected/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-kube-api-access-5phvb\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114137 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-lib-modules\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114137 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-proc\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114137 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-sys\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114137 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-podres\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114307 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114182 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-lib-modules\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114307 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114193 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-proc\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114307 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114199 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-sys\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.114307 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.114247 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-podres\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.122928 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.122905 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5phvb\" (UniqueName: \"kubernetes.io/projected/8e57b6c7-80a3-4c4b-93a4-61d771b84a6f-kube-api-access-5phvb\") pod \"perf-node-gather-daemonset-fhpt4\" (UID: \"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.225368 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.225349 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:57.496816 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.496754 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4"] Sep 17 18:24:57.499738 ip-10-0-128-248 kubenswrapper[2583]: W0917 18:24:57.499708 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e57b6c7_80a3_4c4b_93a4_61d771b84a6f.slice/crio-145b8177d484f505e5eecfdcb230de459baff905b70a3f4948c91597724218a6 WatchSource:0}: Error finding container 145b8177d484f505e5eecfdcb230de459baff905b70a3f4948c91597724218a6: Status 404 returned error can't find the container with id 145b8177d484f505e5eecfdcb230de459baff905b70a3f4948c91597724218a6 Sep 17 18:24:57.667718 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.667698 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bsbv6_2b60746a-01ad-4609-a3ef-e31bbdf74407/dns/0.log" Sep 17 18:24:57.687875 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.687855 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bsbv6_2b60746a-01ad-4609-a3ef-e31bbdf74407/kube-rbac-proxy/0.log" Sep 17 18:24:57.799170 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:57.799152 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s9gfm_2f1a7c3c-7d1b-41c2-bc7f-28634f1234a0/dns-node-resolver/0.log" Sep 17 18:24:58.366700 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:58.366678 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97t8m_1f9ccc9b-6d34-4923-8320-ad4586f8f1d0/node-ca/0.log" Sep 17 18:24:58.373758 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:58.373734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" event={"ID":"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f","Type":"ContainerStarted","Data":"371812442e1777fd70d9a4496f8cbb25e06f8778fc62f01d8cd9b3169aaebd02"} Sep 17 18:24:58.373854 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:58.373786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" event={"ID":"8e57b6c7-80a3-4c4b-93a4-61d771b84a6f","Type":"ContainerStarted","Data":"145b8177d484f505e5eecfdcb230de459baff905b70a3f4948c91597724218a6"} Sep 17 18:24:58.373854 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:58.373810 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:24:58.392448 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:58.392404 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" podStartSLOduration=2.39239069 podStartE2EDuration="2.39239069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 18:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:24:58.391069719 +0000 UTC m=+3047.637110522" watchObservedRunningTime="2026-09-17 18:24:58.39239069 +0000 UTC m=+3047.638431494" Sep 17 18:24:59.164447 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.164426 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f58f99974-897ql_4207130c-c9e5-4358-9860-7f2ebffa6615/router/0.log" Sep 17 18:24:59.540079 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.540054 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qjmqs_e681489d-a82b-44a3-bade-59fe825f1260/serve-healthcheck-canary/0.log" Sep 17 18:24:59.867269 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.867143 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-spbxm_47f5f068-c587-41ea-b827-a1736453bb8d/insights-operator/0.log" Sep 17 18:24:59.867970 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.867944 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-spbxm_47f5f068-c587-41ea-b827-a1736453bb8d/insights-operator/1.log" Sep 17 18:24:59.886878 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.886859 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-224km_54552639-5771-4e16-b7e4-23b4335e4faa/kube-rbac-proxy/0.log" Sep 17 18:24:59.905701 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.905680 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-224km_54552639-5771-4e16-b7e4-23b4335e4faa/exporter/0.log" Sep 17 18:24:59.925933 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:24:59.925912 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-224km_54552639-5771-4e16-b7e4-23b4335e4faa/extractor/0.log" Sep 17 18:25:04.388267 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:04.388238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-fhpt4" Sep 17 18:25:07.044122 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.044094 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-4h7d8_46a12da8-7e00-40c0-a736-3071a72c4147/kube-storage-version-migrator-operator/1.log" Sep 17 18:25:07.045231 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.045205 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-4h7d8_46a12da8-7e00-40c0-a736-3071a72c4147/kube-storage-version-migrator-operator/0.log" Sep 17 18:25:07.896972 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.896948 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/kube-multus-additional-cni-plugins/0.log" Sep 17 18:25:07.918080 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.918058 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/egress-router-binary-copy/0.log" Sep 17 18:25:07.942795 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.942778 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/cni-plugins/0.log" Sep 17 18:25:07.964221 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.964202 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/bond-cni-plugin/0.log" Sep 17 18:25:07.990646 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:07.990605 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/routeoverride-cni/0.log" Sep 17 18:25:08.010935 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:08.010918 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/whereabouts-cni-bincopy/0.log" Sep 17 18:25:08.030054 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:08.030032 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44pd9_ab89a7e9-37cb-46e9-8535-e1d6173de675/whereabouts-cni/0.log" Sep 17 18:25:08.385063 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:08.385044 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ds6zl_c3851eaf-b94a-4016-a49b-b45a5efb0c0f/kube-multus/0.log" Sep 17 18:25:08.477980 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:08.477957 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nkskg_cdd47ebc-76f8-4ff4-92a6-39f2700c35be/network-metrics-daemon/0.log" Sep 17 18:25:08.505087 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:08.505068 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nkskg_cdd47ebc-76f8-4ff4-92a6-39f2700c35be/kube-rbac-proxy/0.log" Sep 17 18:25:09.638060 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.638030 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/ovn-controller/0.log" Sep 17 18:25:09.657616 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.657559 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/ovn-acl-logging/0.log" Sep 17 18:25:09.679349 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.679327 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/kube-rbac-proxy-node/0.log" Sep 17 18:25:09.697607 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.697580 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 18:25:09.714613 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.714591 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/northd/0.log" Sep 17 18:25:09.733296 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.733277 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/nbdb/0.log" Sep 17 18:25:09.754496 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.754477 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/sbdb/0.log" Sep 17 18:25:09.792944 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:09.792929 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmrg9_d2429979-1441-417e-8241-16e35406a3a4/ovnkube-controller/0.log" Sep 17 18:25:10.784564 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:10.784537 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-2kvhf_b914f895-ad9f-4472-88e3-2de946cf4a9f/check-endpoints/0.log" Sep 17 18:25:10.852457 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:10.852429 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-lxqq9_51140b5c-ad9c-40c4-8e15-572191352895/network-check-target-container/0.log" Sep 17 18:25:11.683280 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:11.683231 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-426rl_a041459a-331e-4af8-a415-9f072005e2ec/iptables-alerter/0.log" Sep 17 18:25:12.331179 ip-10-0-128-248 kubenswrapper[2583]: I0917 18:25:12.331154 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-fbxm5_610a5ea2-1a65-4740-8366-66869cc3ba6b/tuned/0.log"