Sep 09 23:58:16.895983 ip-10-0-128-28 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 23:58:16.895993 ip-10-0-128-28 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 23:58:16.896003 ip-10-0-128-28 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 23:58:16.896382 ip-10-0-128-28 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 23:58:26.283252 ip-10-0-128-28 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 23:58:26.283265 ip-10-0-128-28 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0b066760a6814d3bbafd27062f764cbd -- Sep 09 23:59:57.208831 ip-10-0-128-28 systemd[1]: Starting Kubernetes Kubelet... Sep 09 23:59:57.634202 ip-10-0-128-28 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:59:57.634202 ip-10-0-128-28 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 23:59:57.634202 ip-10-0-128-28 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:59:57.634202 ip-10-0-128-28 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 23:59:57.634202 ip-10-0-128-28 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:59:57.635329 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.635239 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 23:59:57.640187 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640049 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.640187 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640183 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.640187 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640186 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.640187 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640192 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.640187 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640195 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640198 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640202 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640205 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640207 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640210 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640213 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640216 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640219 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640221 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640225 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640228 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640231 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640233 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640236 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640239 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640242 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640245 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640247 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640250 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.640376 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640252 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640255 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640258 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640260 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640263 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640266 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640268 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640273 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640277 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640279 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640282 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640284 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640287 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640290 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640293 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640296 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640299 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640301 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640304 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.640868 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640307 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640309 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640314 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640317 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640320 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640323 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640327 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640330 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640333 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640336 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640339 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640342 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640345 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640348 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640350 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640353 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640355 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640358 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640361 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.641416 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640363 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640366 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640368 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640371 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640373 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640376 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640379 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640381 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640386 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640389 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640391 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640394 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640397 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640399 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640402 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640405 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640407 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640410 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640412 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640415 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.641926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640418 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640421 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640423 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640426 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640429 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640431 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640434 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640437 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640441 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640443 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640446 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640449 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640451 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640454 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640456 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640459 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.640464 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641236 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641242 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641246 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.642422 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641249 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641253 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641255 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641258 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641261 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641263 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641266 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641269 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641273 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641277 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641280 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641283 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641286 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641289 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641293 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641296 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641299 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641301 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641305 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641308 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.642926 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641310 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641313 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641316 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641319 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641322 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641325 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641327 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641330 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641332 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641336 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641338 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641341 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641344 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641347 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641350 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641352 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641355 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641358 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641361 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.643414 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641364 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641366 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641369 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641372 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641374 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641376 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641379 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641382 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641385 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641388 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641390 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641393 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641397 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641401 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641404 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641406 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641409 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641411 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641414 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.643905 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641417 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641419 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641422 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641425 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641428 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641431 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641437 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641442 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641445 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641448 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641451 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641455 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641457 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641460 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641462 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641465 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641468 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641470 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641473 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.644368 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641475 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641477 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641480 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641484 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641487 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641489 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641492 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641495 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641497 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641500 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641502 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641505 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641507 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641510 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641530 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641535 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641539 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641542 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641545 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641910 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.644844 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641914 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641917 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641920 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641923 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641925 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641928 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641931 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641933 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641936 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641939 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641943 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641947 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641950 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641955 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641958 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641961 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641964 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641967 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641970 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.645330 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641972 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641975 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641978 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641980 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641983 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641986 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641988 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641991 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641994 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641996 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.641999 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642001 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642005 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642007 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642010 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642013 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642016 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642018 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642021 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642024 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.645798 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642026 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642029 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642031 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642034 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642036 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642039 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642041 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642044 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642046 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642049 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642051 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642054 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642056 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642059 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642062 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642064 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642067 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642069 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642071 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.646275 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642074 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642076 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642080 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642082 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642086 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642090 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642093 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642095 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642098 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642101 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642104 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642106 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642109 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642111 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642114 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642116 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642119 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642122 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642124 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642127 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.646797 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642129 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642133 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642135 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642138 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642141 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642143 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642146 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642148 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642151 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642154 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642156 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642159 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642161 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642164 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642167 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642169 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642174 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642177 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642179 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642182 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642253 2582 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 23:59:57.647282 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642288 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642294 2582 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642300 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642304 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642307 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642311 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642316 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642319 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642322 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642326 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642329 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642332 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642335 2582 flags.go:64] FLAG: --cgroup-root="" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642338 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642341 2582 flags.go:64] FLAG: --client-ca-file="" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642345 2582 flags.go:64] FLAG: --cloud-provider="external" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642348 2582 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642352 2582 flags.go:64] FLAG: --cluster-domain="" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642355 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642358 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642361 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642365 2582 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642368 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 23:59:57.647805 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642371 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642374 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642378 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642380 2582 flags.go:64] FLAG: --contention-profiling="false" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642385 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642388 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642391 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642394 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642398 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642401 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642404 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642407 2582 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642410 2582 flags.go:64] FLAG: --enable-server="true" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642412 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642416 2582 flags.go:64] FLAG: --event-burst="100" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642419 2582 flags.go:64] FLAG: --event-qps="50" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642422 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642426 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642429 2582 flags.go:64] FLAG: --eviction-hard="" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642433 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642436 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642439 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642442 2582 flags.go:64] FLAG: --eviction-soft="" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642445 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642448 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 23:59:57.648353 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642451 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642454 2582 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642457 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642460 2582 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642463 2582 flags.go:64] FLAG: --feature-gates="" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642466 2582 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642470 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642472 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642475 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642478 2582 flags.go:64] FLAG: --healthz-port="10248" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642481 2582 flags.go:64] FLAG: --help="false" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642485 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642491 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642494 2582 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642497 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642501 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642504 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642507 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642510 2582 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642526 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642530 2582 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642533 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642536 2582 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642539 2582 flags.go:64] FLAG: --kube-reserved="" Sep 09 23:59:57.648981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642542 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642545 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642548 2582 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642551 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642554 2582 flags.go:64] FLAG: --lock-file="" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642557 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642560 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642563 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642568 2582 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642572 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642575 2582 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642578 2582 flags.go:64] FLAG: --logging-format="text" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642581 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642584 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642587 2582 flags.go:64] FLAG: --manifest-url="" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642590 2582 flags.go:64] FLAG: --manifest-url-header="" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642594 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642597 2582 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642601 2582 flags.go:64] FLAG: --max-pods="110" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642605 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642609 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642612 2582 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642615 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642618 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642621 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 23:59:57.649587 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642624 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642632 2582 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642635 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642638 2582 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642641 2582 flags.go:64] FLAG: --pod-cidr="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642644 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642650 2582 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642653 2582 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642666 2582 flags.go:64] FLAG: --pods-per-core="0" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642669 2582 flags.go:64] FLAG: --port="10250" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642673 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642676 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0408b70b6cc1e2b41" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642680 2582 flags.go:64] FLAG: --qos-reserved="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642682 2582 flags.go:64] FLAG: --read-only-port="10255" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642685 2582 flags.go:64] FLAG: --register-node="true" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642688 2582 flags.go:64] FLAG: --register-with-taints="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642692 2582 flags.go:64] FLAG: --registry-burst="10" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642696 2582 flags.go:64] FLAG: --registry-qps="5" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642699 2582 flags.go:64] FLAG: --reserved-cpus="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642701 2582 flags.go:64] FLAG: --reserved-memory="" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642705 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642708 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642711 2582 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642714 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642717 2582 flags.go:64] FLAG: --runonce="false" Sep 09 23:59:57.650250 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642719 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642724 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642727 2582 flags.go:64] FLAG: --seccomp-default="false" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642731 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642734 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642737 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642740 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642744 2582 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642746 2582 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642749 2582 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642752 2582 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642755 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642758 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642761 2582 flags.go:64] FLAG: --system-cgroups="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642764 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642770 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642772 2582 flags.go:64] FLAG: --tls-cert-file="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642775 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642780 2582 flags.go:64] FLAG: --tls-min-version="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642783 2582 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642785 2582 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642788 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642791 2582 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642794 2582 flags.go:64] FLAG: --v="2" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642798 2582 flags.go:64] FLAG: --version="false" Sep 09 23:59:57.650896 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642802 2582 flags.go:64] FLAG: --vmodule="" Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642806 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.642809 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642949 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642953 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642956 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642959 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642962 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642967 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642970 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642974 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642976 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642979 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642981 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642984 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642988 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642990 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642993 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642995 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.651524 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.642999 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643004 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643008 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643011 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643014 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643017 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643020 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643024 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643026 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643029 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643032 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643035 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643038 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643040 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643046 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643048 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643051 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643054 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643057 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.652008 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643059 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643062 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643065 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643068 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643071 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643074 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643077 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643079 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643082 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643084 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643087 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643091 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643094 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643098 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643100 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643103 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643106 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643109 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643112 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643115 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.652468 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643118 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643120 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643123 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643126 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643129 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643132 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643135 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643137 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643141 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643143 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643146 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643148 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643151 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643153 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643157 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643160 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643164 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643166 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643169 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643172 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.652971 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643174 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643177 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643179 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643182 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643184 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643187 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643190 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643192 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643195 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643197 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643200 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643202 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643205 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643207 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643210 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643212 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643215 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643217 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643220 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643222 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.653489 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643225 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643228 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643230 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.643233 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.643948 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.651602 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.651620 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.652470 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.652478 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652549 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652555 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652558 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652562 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652565 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652568 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.653997 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652571 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652573 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652576 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652581 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652584 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652587 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652590 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652592 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652595 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652598 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652601 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652603 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652606 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652609 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652611 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652614 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652617 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652619 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652622 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652625 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.654366 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652627 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652630 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652633 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652637 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652639 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652642 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652645 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652647 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652650 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652652 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652655 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652658 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652660 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652665 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652669 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652672 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652674 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652677 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652679 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.654875 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652682 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652685 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652687 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652690 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652693 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652696 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652698 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652701 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652704 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652706 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652709 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652712 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652715 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652717 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652720 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652722 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652725 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652728 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652730 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.655347 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652733 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652735 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652738 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652740 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652743 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652745 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652748 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652750 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652753 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652755 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652758 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652760 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652763 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652765 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652768 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652770 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652773 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652776 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652779 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652781 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.655904 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652784 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652787 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652789 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652792 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652794 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652798 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652801 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652804 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652807 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652810 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652813 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652816 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652819 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652821 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652824 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.656387 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.652829 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652943 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652948 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652950 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652953 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652956 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652959 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652962 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652964 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652967 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652970 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652973 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652975 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652978 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652982 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652984 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652987 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652990 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652993 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652995 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:59:57.656781 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.652998 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653001 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653003 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653006 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653008 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653011 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653013 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653016 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653019 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653022 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653025 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653028 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653031 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653033 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653036 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653038 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653041 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653043 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653046 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653049 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:59:57.657268 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653051 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653054 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653056 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653059 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653062 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653064 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653067 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653069 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653072 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653074 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653077 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653080 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653082 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653085 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653087 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653090 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653092 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653095 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653097 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653100 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:59:57.657759 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653102 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653105 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653108 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653112 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653115 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653119 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653122 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653124 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653127 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653129 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653132 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653134 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653137 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653139 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653142 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653144 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653147 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653150 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653152 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:59:57.658233 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653155 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653157 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653161 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653165 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653168 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653171 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653173 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653176 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653179 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653181 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653183 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653186 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653188 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653191 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653194 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653197 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653200 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653202 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653205 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:59:57.658716 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653207 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:57.653211 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.653216 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.653333 2582 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.655674 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.656756 2582 server.go:1019] "Starting client certificate rotation" Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.656853 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 23:59:57.659223 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.656914 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 23:59:57.660074 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.660060 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 23:59:57.661202 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.661186 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 23:59:57.671338 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.671313 2582 log.go:25] "Validated CRI v1 runtime API" Sep 09 23:59:57.674180 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.674164 2582 log.go:25] "Validated CRI v1 image API" Sep 09 23:59:57.675021 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.674997 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 23:59:57.678364 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.678344 2582 fs.go:135] Filesystem UUIDs: map[1ec75efa-f6db-4dee-ba15-0d22f9af554e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8c1f487e-bfe4-4cc7-a325-c15912bc19aa:/dev/nvme0n1p3] Sep 09 23:59:57.678410 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.678365 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 23:59:57.684490 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.684381 2582 manager.go:216] Machine: {Timestamp:2026-09-09 23:59:57.683012934 +0000 UTC m=+0.363945683 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099388 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2847063bc89195fa1613a06b655156 SystemUUID:ec284706-3bc8-9195-fa16-13a06b655156 BootID:0b066760-a681-4d3b-bafd-27062f764cbd Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4e:3c:c5:80:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4e:3c:c5:80:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:9a:e7:c6:84:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 23:59:57.684490 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.684489 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 23:59:57.684606 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.684590 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 23:59:57.688194 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.688165 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 23:59:57.688366 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.688197 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 23:59:57.689633 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.689621 2582 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 23:59:57.689669 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.689635 2582 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 23:59:57.689669 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.689644 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 23:59:57.690269 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.690253 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 23:59:57.690416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.690405 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 23:59:57.692100 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.692081 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 23:59:57.695471 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695459 2582 state_mem.go:36] "Initialized new in-memory state store" Sep 09 23:59:57.695612 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695602 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 23:59:57.695653 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695634 2582 kubelet.go:486] "Attempting to sync node with API server" Sep 09 23:59:57.695653 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695644 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 23:59:57.695713 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695659 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 23:59:57.695713 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695667 2582 kubelet.go:398] "Adding apiserver pod source" Sep 09 23:59:57.695713 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.695679 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 23:59:57.696022 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.696010 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 23:59:57.696083 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.696028 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 23:59:57.699434 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.699415 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 23:59:57.702887 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.702867 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 23:59:57.702964 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.702901 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 23:59:57.703054 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703044 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 23:59:57.703092 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703058 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 23:59:57.703092 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703064 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 23:59:57.703092 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703070 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 23:59:57.703092 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703077 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 23:59:57.703092 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703083 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 23:59:57.703790 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703781 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 23:59:57.703790 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703790 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 23:59:57.703846 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703798 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 23:59:57.703846 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703804 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 23:59:57.703846 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703812 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 23:59:57.703846 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.703821 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 23:59:57.704613 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.704602 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 23:59:57.704613 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.704613 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 23:59:57.706817 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.706773 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 23:59:57.706919 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.706822 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 23:59:57.706919 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.706881 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 23:59:57.708634 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.708618 2582 server.go:1268] "Started kubelet" Sep 09 23:59:57.708733 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.708672 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 23:59:57.708876 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.708825 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 23:59:57.708960 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.708874 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 23:59:57.709677 ip-10-0-128-28 systemd[1]: Started Kubernetes Kubelet. Sep 09 23:59:57.710800 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.710779 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 23:59:57.716048 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.716025 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 23:59:57.717336 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.717318 2582 server.go:310] "Adding debug handlers to kubelet server" Sep 09 23:59:57.720039 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.720013 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 23:59:57.720801 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.720779 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 23:59:57.720924 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.720902 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:57.721656 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.721635 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 23:59:57.721656 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.721637 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 23:59:57.721769 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.721669 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 23:59:57.721769 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.721736 2582 reconstruct.go:97] "Volume reconstruction finished" Sep 09 23:59:57.721769 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.721745 2582 reconciler.go:29] "Reconciler: start to sync state" Sep 09 23:59:57.722024 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722006 2582 factory.go:153] Registering CRI-O factory Sep 09 23:59:57.722101 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722028 2582 factory.go:223] Registration of the crio container factory successfully Sep 09 23:59:57.722101 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722084 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 23:59:57.722101 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722096 2582 factory.go:55] Registering systemd factory Sep 09 23:59:57.722219 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722105 2582 factory.go:223] Registration of the systemd container factory successfully Sep 09 23:59:57.722219 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722123 2582 factory.go:103] Registering Raw factory Sep 09 23:59:57.722219 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722140 2582 manager.go:1196] Started watching for new ooms in manager Sep 09 23:59:57.722676 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.722661 2582 manager.go:349] Starting recovery of all containers Sep 09 23:59:57.726701 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.726641 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 23:59:57.727359 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.727161 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 23:59:57.728085 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.726728 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-28.ec2.internal.18d3cc906f298872 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-28.ec2.internal,UID:ip-10-0-128-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-28.ec2.internal,},FirstTimestamp:2026-09-09 23:59:57.70856869 +0000 UTC m=+0.389501423,LastTimestamp:2026-09-09 23:59:57.70856869 +0000 UTC m=+0.389501423,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-28.ec2.internal,}" Sep 09 23:59:57.730906 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.730884 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rggv2" Sep 09 23:59:57.733304 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.733287 2582 manager.go:354] Recovery completed Sep 09 23:59:57.737438 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.737414 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rggv2" Sep 09 23:59:57.738252 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.738240 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.740653 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.740637 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.740708 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.740671 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.740708 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.740685 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.741283 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.741269 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 23:59:57.741283 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.741281 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 23:59:57.741390 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.741287 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.741390 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.741298 2582 state_mem.go:36] "Initialized new in-memory state store" Sep 09 23:59:57.742066 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742049 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.742142 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742083 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.742142 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742095 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.742429 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742413 2582 policy_none.go:49] "None policy: Start" Sep 09 23:59:57.742510 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742435 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 23:59:57.742510 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.742449 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 23:59:57.742696 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.742580 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-28.ec2.internal.18d3cc9071131b92 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-28.ec2.internal,UID:ip-10-0-128-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-28.ec2.internal,},FirstTimestamp:2026-09-09 23:59:57.740653458 +0000 UTC m=+0.421586191,LastTimestamp:2026-09-09 23:59:57.740653458 +0000 UTC m=+0.421586191,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-28.ec2.internal,}" Sep 09 23:59:57.743456 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.743444 2582 policy_none.go:47] "Start" Sep 09 23:59:57.779180 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779157 2582 manager.go:337] "Starting Device Plugin manager" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.779205 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779217 2582 server.go:87] "Starting device plugin registration server" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779611 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779624 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779863 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779947 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.779956 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.780397 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 23:59:57.783719 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.780450 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:57.820159 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.820133 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 23:59:57.821307 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.821291 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 23:59:57.821367 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.821343 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 23:59:57.821367 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.821366 2582 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 23:59:57.821486 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.821466 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 23:59:57.823398 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.823383 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:59:57.880270 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.880242 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.881396 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.881380 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.881462 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.881413 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.881462 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.881428 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.881462 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.881452 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.890017 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.889958 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.890017 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.889980 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-28.ec2.internal\": node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:57.910631 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.910604 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:57.921656 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.921632 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal"] Sep 09 23:59:57.921763 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.921749 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.921897 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.921880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.921960 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.921907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.922728 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.922708 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.922839 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.922737 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.922839 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.922747 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.924889 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.924875 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.925025 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.925090 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925048 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.925651 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925636 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.925727 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925661 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.925727 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925674 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.925727 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925638 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.925727 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925728 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.925913 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.925738 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.927779 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.927760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.927864 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.927786 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:59:57.930720 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.930702 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:59:57.930812 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.930737 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:59:57.930812 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:57.930750 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:59:57.944123 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.944099 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-28.ec2.internal\" not found" node="ip-10-0-128-28.ec2.internal" Sep 09 23:59:57.949060 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:57.949042 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-28.ec2.internal\" not found" node="ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.011481 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.011443 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.022778 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.022743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.022778 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.022779 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.022944 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.022841 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.022944 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.022844 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d8773881ab781b00886710f6952dd2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal\" (UID: \"d2d8773881ab781b00886710f6952dd2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.111925 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.111887 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.123167 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.123141 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a426b2798cb1a9b52d39c4792ff821c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-28.ec2.internal\" (UID: \"1a426b2798cb1a9b52d39c4792ff821c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.212632 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.212565 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.223943 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.223916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a426b2798cb1a9b52d39c4792ff821c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-28.ec2.internal\" (UID: \"1a426b2798cb1a9b52d39c4792ff821c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.224001 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.223966 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a426b2798cb1a9b52d39c4792ff821c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-28.ec2.internal\" (UID: \"1a426b2798cb1a9b52d39c4792ff821c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.246745 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.246706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.251874 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.251849 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.313216 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.313181 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.420612 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.420578 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.521171 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.521082 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.628778 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.628739 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-28.ec2.internal\" not found" Sep 09 23:59:58.647671 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.647640 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:59:58.656667 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.656642 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 23:59:58.656792 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.656770 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 23:59:58.656851 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.656810 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 23:59:58.696005 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.695967 2582 apiserver.go:52] "Watching apiserver" Sep 09 23:59:58.712644 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.712615 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 23:59:58.715062 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.715035 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-4dcbj","openshift-ovn-kubernetes/ovnkube-node-w8rrn","kube-system/global-pull-secret-syncer-fc4mk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd","openshift-cluster-node-tuning-operator/tuned-5xbj4","openshift-image-registry/node-ca-p977b","openshift-multus/multus-additional-cni-plugins-ldwtj","openshift-multus/multus-s8dlv","kube-system/konnectivity-agent-t8h24","openshift-dns/node-resolver-5g9z9","openshift-multus/network-metrics-daemon-bxlwt","openshift-network-diagnostics/network-check-target-7cqgr"] Sep 09 23:59:58.717516 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.717496 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.719956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.719934 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.720069 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.719936 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 23:59:58.720143 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.720134 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 23:59:58.720251 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.720228 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4zmz\"" Sep 09 23:59:58.720626 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.720444 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.720626 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.720621 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.722253 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.721576 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.723686 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.723664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.725532 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.724552 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 23:59:58.725532 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.724952 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 23:59:58.725903 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.725861 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.727078 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.726270 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.727078 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.726442 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 23:59:58.727078 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.726633 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.727078 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.726741 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 23:59:58.727078 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.726832 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n6z72\"" Sep 09 23:59:58.727394 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.727201 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 23:59:58.728052 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.727970 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.728175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728153 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6hm4t\"" Sep 09 23:59:58.728555 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728536 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.728842 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grv8h\" (UniqueName: \"kubernetes.io/projected/4dd25809-0f6c-4ff3-86af-c634bbea8253-kube-api-access-grv8h\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.728948 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-systemd-units\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.728948 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-var-lib-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.728948 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728894 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-node-log\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.728948 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728958 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-sys-fs\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.728985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebfc1af-7996-405e-a28c-740cff312cdb-host-slash\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsns6\" (UniqueName: \"kubernetes.io/projected/3ebfc1af-7996-405e-a28c-740cff312cdb-kube-api-access-dsns6\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-slash\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729082 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-etc-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-log-socket\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729174 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-netd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovn-node-metrics-cert\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-netns\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729275 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-bin\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729323 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-registration-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729347 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbsmg\" (UniqueName: \"kubernetes.io/projected/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kube-api-access-cbsmg\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-socket-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729396 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-device-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729418 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebfc1af-7996-405e-a28c-740cff312cdb-iptables-alerter-script\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729434 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-systemd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-kubelet\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729479 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-script-lib\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-ovn\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.729573 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-config\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.730150 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.729584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-env-overrides\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.730848 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.730821 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.730848 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.730835 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.730993 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.730857 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wv68c\"" Sep 09 23:59:58.730993 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.730824 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 23:59:58.731378 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.731360 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.732989 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.732970 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 23:59:58.733059 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.732993 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 23:59:58.733059 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.733043 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" Sep 09 23:59:58.734014 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.733994 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2gx9r\"" Sep 09 23:59:58.734105 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.734050 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.734105 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.734063 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.734186 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.734132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 09 23:59:58.734224 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.734201 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 09 23:59:58.737132 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.737112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.737221 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.737178 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 09 23:59:58.739082 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.739045 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 23:54:58 +0000 UTC" deadline="2028-03-26 18:06:05.115978786 +0000 UTC" Sep 09 23:59:58.739082 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.739081 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13530h6m6.37690046s" Sep 09 23:59:58.739807 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.739792 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.742304 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.742284 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.742421 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.742329 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hkv9s\"" Sep 09 23:59:58.742421 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.742291 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.743454 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.743433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.746570 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.746550 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.747038 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747019 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 23:59:58.747124 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747048 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 23:59:58.747124 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747019 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 23:59:58.747255 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747146 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 23:59:58.747255 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747240 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ghxck\"" Sep 09 23:59:58.747374 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.747273 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 23:59:58.749332 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.749313 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.749455 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.749434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wz2jl\"" Sep 09 23:59:58.749455 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.749443 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 23:59:58.750034 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.750018 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 23:59:58.751925 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.751909 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 23:59:58.752100 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.752083 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal"] Sep 09 23:59:58.752183 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.752127 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal"] Sep 09 23:59:58.752251 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.752192 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:58.752320 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.752254 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 09 23:59:58.752320 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.752300 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dqkdl\"" Sep 09 23:59:58.752402 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.752351 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 23:59:58.756629 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.756608 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fvg2j" Sep 09 23:59:58.764391 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.764372 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fvg2j" Sep 09 23:59:58.823763 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.823730 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 23:59:58.830482 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-tmp\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.830635 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-etc-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830635 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-log-socket\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830635 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830561 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-etc-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830635 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-log-socket\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830635 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovn-node-metrics-cert\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-socket-dir-parent\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-multus\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830715 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-bin\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbsmg\" (UniqueName: \"kubernetes.io/projected/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kube-api-access-cbsmg\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830797 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830819 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-conf\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-kubelet\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 09 23:59:58.830907 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830906 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysconfig\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830933 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-bin\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1749c27-f505-440c-924a-9952ec646773-serviceca\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.830978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-systemd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1749c27-f505-440c-924a-9952ec646773-host\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-k8s-cni-cncf-io\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-hostroot\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831057 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-systemd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-conf-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831068 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831123 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-multus-certs\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-sys\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831214 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-netns\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831269 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eab51a-ba59-4c67-bf32-89795665a505-agent-certs\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grv8h\" (UniqueName: \"kubernetes.io/projected/4dd25809-0f6c-4ff3-86af-c634bbea8253-kube-api-access-grv8h\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7snvl\" (UniqueName: \"kubernetes.io/projected/2cb4843b-acb3-45bd-98c1-11acccb93f3a-kube-api-access-7snvl\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831404 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.831416 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnrc\" (UniqueName: \"kubernetes.io/projected/40e30186-ac33-4218-9719-62255d7f2414-kube-api-access-lsnrc\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-var-lib-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831518 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-sys-fs\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831628 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831630 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-var-lib-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831680 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-dbus\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831731 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-sys-fs\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831774 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-modprobe-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831814 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cni-binary-copy\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831832 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-etc-kubernetes\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eab51a-ba59-4c67-bf32-89795665a505-konnectivity-ca\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebfc1af-7996-405e-a28c-740cff312cdb-host-slash\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsns6\" (UniqueName: \"kubernetes.io/projected/3ebfc1af-7996-405e-a28c-740cff312cdb-kube-api-access-dsns6\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-slash\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-netd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.832212 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831996 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-tuned\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.831992 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebfc1af-7996-405e-a28c-740cff312cdb-host-slash\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-hosts-file\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832082 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-cni-netd\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832116 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-tmp-dir\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832153 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-slash\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832174 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-netns\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-registration-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832207 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-run\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-run-netns\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832251 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832265 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-registration-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-system-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-socket-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832330 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-device-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833130 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-kubernetes\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832445 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832469 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-socket-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebfc1af-7996-405e-a28c-740cff312cdb-iptables-alerter-script\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832490 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-device-dir\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832522 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-kubelet-config\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832601 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-kubelet\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832658 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-script-lib\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832665 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-openvswitch\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-systemd\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832705 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-host-kubelet\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-var-lib-kubelet\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-host\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js5h4\" (UniqueName: \"kubernetes.io/projected/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-kube-api-access-js5h4\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.833956 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp7b4\" (UniqueName: \"kubernetes.io/projected/f1749c27-f505-440c-924a-9952ec646773-kube-api-access-xp7b4\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-system-cni-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-ovn\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-config\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-env-overrides\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.832989 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-binary-copy\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-bin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crbbj\" (UniqueName: \"kubernetes.io/projected/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-kube-api-access-crbbj\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-systemd-units\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833084 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebfc1af-7996-405e-a28c-740cff312cdb-iptables-alerter-script\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833093 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-node-log\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833147 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cnibin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833159 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-script-lib\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llww\" (UniqueName: \"kubernetes.io/projected/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-kube-api-access-4llww\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-lib-modules\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.834865 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-cnibin\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-daemon-config\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833391 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-systemd-units\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833443 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-node-log\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833502 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-env-overrides\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833530 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5f342fa-ab62-494b-aba9-00bcfa4e6551-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833585 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dd25809-0f6c-4ff3-86af-c634bbea8253-run-ovn\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.833656 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovnkube-config\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.835831 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.834800 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dd25809-0f6c-4ff3-86af-c634bbea8253-ovn-node-metrics-cert\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.839295 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.839268 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbsmg\" (UniqueName: \"kubernetes.io/projected/d5f342fa-ab62-494b-aba9-00bcfa4e6551-kube-api-access-cbsmg\") pod \"aws-ebs-csi-driver-node-xrspd\" (UID: \"d5f342fa-ab62-494b-aba9-00bcfa4e6551\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:58.839799 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.839773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsns6\" (UniqueName: \"kubernetes.io/projected/3ebfc1af-7996-405e-a28c-740cff312cdb-kube-api-access-dsns6\") pod \"iptables-alerter-4dcbj\" (UID: \"3ebfc1af-7996-405e-a28c-740cff312cdb\") " pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:58.840023 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.840003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grv8h\" (UniqueName: \"kubernetes.io/projected/4dd25809-0f6c-4ff3-86af-c634bbea8253-kube-api-access-grv8h\") pod \"ovnkube-node-w8rrn\" (UID: \"4dd25809-0f6c-4ff3-86af-c634bbea8253\") " pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:58.886820 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.886791 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:59:58.909939 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:58.909882 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a426b2798cb1a9b52d39c4792ff821c.slice/crio-d951f9ef34c291e0a85f8015f20e564be9d7bf13ba7ae5842c81203aaf01763d WatchSource:0}: Error finding container d951f9ef34c291e0a85f8015f20e564be9d7bf13ba7ae5842c81203aaf01763d: Status 404 returned error can't find the container with id d951f9ef34c291e0a85f8015f20e564be9d7bf13ba7ae5842c81203aaf01763d Sep 09 23:59:58.913267 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.913246 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:59:58.933952 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.933919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.933952 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.933953 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cnibin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.933971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llww\" (UniqueName: \"kubernetes.io/projected/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-kube-api-access-4llww\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.933989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-lib-modules\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934065 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cnibin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934084 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-lib-modules\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-cnibin\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934100 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-cnibin\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-daemon-config\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934175 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934154 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-tmp\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-socket-dir-parent\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934210 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-multus\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-conf\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-socket-dir-parent\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934287 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-multus\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-kubelet\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysconfig\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1749c27-f505-440c-924a-9952ec646773-serviceca\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934392 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-conf\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934430 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-kubelet\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1749c27-f505-440c-924a-9952ec646773-host\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-k8s-cni-cncf-io\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-hostroot\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-conf-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-multus-certs\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-k8s-cni-cncf-io\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.934660 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934576 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-sys\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934615 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1749c27-f505-440c-924a-9952ec646773-host\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934623 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-netns\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934651 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eab51a-ba59-4c67-bf32-89795665a505-agent-certs\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934663 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-conf-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934702 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7snvl\" (UniqueName: \"kubernetes.io/projected/2cb4843b-acb3-45bd-98c1-11acccb93f3a-kube-api-access-7snvl\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934713 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934722 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-multus-certs\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-sys\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnrc\" (UniqueName: \"kubernetes.io/projected/40e30186-ac33-4218-9719-62255d7f2414-kube-api-access-lsnrc\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934789 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-daemon-config\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysconfig\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934818 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-hostroot\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-dbus\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-modprobe-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.934937 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:59:58.935502 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.934984 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-run-netns\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.935010 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-09 23:59:59.434979782 +0000 UTC m=+2.115912505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.935026 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-dbus\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.935076 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-09 23:59:59.435059848 +0000 UTC m=+2.115992590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-modprobe-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cni-binary-copy\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-etc-kubernetes\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eab51a-ba59-4c67-bf32-89795665a505-konnectivity-ca\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-tuned\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935422 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-etc-kubernetes\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-hosts-file\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-tmp-dir\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1749c27-f505-440c-924a-9952ec646773-serviceca\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935699 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-hosts-file\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935808 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-tmp-dir\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-run\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.936492 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935849 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935861 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-cni-binary-copy\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-system-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-run\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-kubernetes\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-system-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-kubernetes\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.935974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-kubelet-config\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-multus-cni-dir\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936067 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eab51a-ba59-4c67-bf32-89795665a505-konnectivity-ca\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936077 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-sysctl-d\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936079 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5068c29a-75f0-4876-b3cb-7de73ef69a18-kubelet-config\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936033 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-systemd\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-var-lib-kubelet\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937195 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936132 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-systemd\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-host\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936183 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-var-lib-kubelet\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936188 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js5h4\" (UniqueName: \"kubernetes.io/projected/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-kube-api-access-js5h4\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936222 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp7b4\" (UniqueName: \"kubernetes.io/projected/f1749c27-f505-440c-924a-9952ec646773-kube-api-access-xp7b4\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-system-cni-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-binary-copy\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936224 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-host\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-bin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936346 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-host-var-lib-cni-bin\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936376 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crbbj\" (UniqueName: \"kubernetes.io/projected/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-kube-api-access-crbbj\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e30186-ac33-4218-9719-62255d7f2414-system-cni-dir\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.936927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.937155 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e30186-ac33-4218-9719-62255d7f2414-cni-binary-copy\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.937772 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.937253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-tmp\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.938237 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.937813 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eab51a-ba59-4c67-bf32-89795665a505-agent-certs\") pod \"konnectivity-agent-t8h24\" (UID: \"25eab51a-ba59-4c67-bf32-89795665a505\") " pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:58.938237 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.938198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-etc-tuned\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.944891 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.944864 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:59:58.945017 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.944899 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:59:58.945017 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.944918 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:59:58.945017 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:58.944986 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-09 23:59:59.444966446 +0000 UTC m=+2.125899192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:59:58.945981 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.945927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llww\" (UniqueName: \"kubernetes.io/projected/dd022f5b-cc5a-409f-89f9-6fe1d42e5748-kube-api-access-4llww\") pod \"node-resolver-5g9z9\" (UID: \"dd022f5b-cc5a-409f-89f9-6fe1d42e5748\") " pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:58.947105 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.947073 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp7b4\" (UniqueName: \"kubernetes.io/projected/f1749c27-f505-440c-924a-9952ec646773-kube-api-access-xp7b4\") pod \"node-ca-p977b\" (UID: \"f1749c27-f505-440c-924a-9952ec646773\") " pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:58.947216 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.947191 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnrc\" (UniqueName: \"kubernetes.io/projected/40e30186-ac33-4218-9719-62255d7f2414-kube-api-access-lsnrc\") pod \"multus-additional-cni-plugins-ldwtj\" (UID: \"40e30186-ac33-4218-9719-62255d7f2414\") " pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:58.947329 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.947309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js5h4\" (UniqueName: \"kubernetes.io/projected/114ee629-8d9c-490b-a6a0-1e4780b4d7a6-kube-api-access-js5h4\") pod \"tuned-5xbj4\" (UID: \"114ee629-8d9c-490b-a6a0-1e4780b4d7a6\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:58.947506 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.947486 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crbbj\" (UniqueName: \"kubernetes.io/projected/69c0d046-adc2-4166-9ce8-f2fd42ff33e8-kube-api-access-crbbj\") pod \"multus-s8dlv\" (UID: \"69c0d046-adc2-4166-9ce8-f2fd42ff33e8\") " pod="openshift-multus/multus-s8dlv" Sep 09 23:59:58.947561 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:58.947505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7snvl\" (UniqueName: \"kubernetes.io/projected/2cb4843b-acb3-45bd-98c1-11acccb93f3a-kube-api-access-7snvl\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:59.057923 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.057830 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dcbj" Sep 09 23:59:59.068544 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.068513 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebfc1af_7996_405e_a28c_740cff312cdb.slice/crio-290a61a625e15f45d9986e22b594662711a19d2a9d1fd4f81504f25b9f8507bd WatchSource:0}: Error finding container 290a61a625e15f45d9986e22b594662711a19d2a9d1fd4f81504f25b9f8507bd: Status 404 returned error can't find the container with id 290a61a625e15f45d9986e22b594662711a19d2a9d1fd4f81504f25b9f8507bd Sep 09 23:59:59.074056 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.074038 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 09 23:59:59.082211 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.082185 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd25809_0f6c_4ff3_86af_c634bbea8253.slice/crio-2ea07fbe5ca0fd94c365eddb49a3c46cfb636deddfa2c8a35f3335401da61893 WatchSource:0}: Error finding container 2ea07fbe5ca0fd94c365eddb49a3c46cfb636deddfa2c8a35f3335401da61893: Status 404 returned error can't find the container with id 2ea07fbe5ca0fd94c365eddb49a3c46cfb636deddfa2c8a35f3335401da61893 Sep 09 23:59:59.087839 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.087818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" Sep 09 23:59:59.092744 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.092722 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p977b" Sep 09 23:59:59.097411 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.097390 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f342fa_ab62_494b_aba9_00bcfa4e6551.slice/crio-8e8efca0dda9b61625678d768f6a08673be1e163bade2142bf5f1bee8e27da3c WatchSource:0}: Error finding container 8e8efca0dda9b61625678d768f6a08673be1e163bade2142bf5f1bee8e27da3c: Status 404 returned error can't find the container with id 8e8efca0dda9b61625678d768f6a08673be1e163bade2142bf5f1bee8e27da3c Sep 09 23:59:59.102127 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.102105 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1749c27_f505_440c_924a_9952ec646773.slice/crio-ea85a8e5305fff103199942fc9c21337bf03877a138a55e76ff44c4375aa7fc9 WatchSource:0}: Error finding container ea85a8e5305fff103199942fc9c21337bf03877a138a55e76ff44c4375aa7fc9: Status 404 returned error can't find the container with id ea85a8e5305fff103199942fc9c21337bf03877a138a55e76ff44c4375aa7fc9 Sep 09 23:59:59.103706 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.103693 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5g9z9" Sep 09 23:59:59.114670 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.114648 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd022f5b_cc5a_409f_89f9_6fe1d42e5748.slice/crio-1d65d316cecada125739dd086db019dd507d504a82b11b99d942334a8e2b476d WatchSource:0}: Error finding container 1d65d316cecada125739dd086db019dd507d504a82b11b99d942334a8e2b476d: Status 404 returned error can't find the container with id 1d65d316cecada125739dd086db019dd507d504a82b11b99d942334a8e2b476d Sep 09 23:59:59.120602 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.120569 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" Sep 09 23:59:59.128768 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.128535 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114ee629_8d9c_490b_a6a0_1e4780b4d7a6.slice/crio-fc82c4e1a710e2550869d81227d27fbfdcc8f1b653c7cbed4a3c9406ac637484 WatchSource:0}: Error finding container fc82c4e1a710e2550869d81227d27fbfdcc8f1b653c7cbed4a3c9406ac637484: Status 404 returned error can't find the container with id fc82c4e1a710e2550869d81227d27fbfdcc8f1b653c7cbed4a3c9406ac637484 Sep 09 23:59:59.134564 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.134547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" Sep 09 23:59:59.142914 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.142893 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e30186_ac33_4218_9719_62255d7f2414.slice/crio-a5fa1a8da9e7aff8a3bdfa9827f7e68c128487d9650ad76ab86e7f61c05f2bbc WatchSource:0}: Error finding container a5fa1a8da9e7aff8a3bdfa9827f7e68c128487d9650ad76ab86e7f61c05f2bbc: Status 404 returned error can't find the container with id a5fa1a8da9e7aff8a3bdfa9827f7e68c128487d9650ad76ab86e7f61c05f2bbc Sep 09 23:59:59.146762 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.146745 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:59:59.153623 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.153607 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s8dlv" Sep 09 23:59:59.158624 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.158605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t8h24" Sep 09 23:59:59.162896 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.162873 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c0d046_adc2_4166_9ce8_f2fd42ff33e8.slice/crio-129fa0d0a58b86f9cca8f580e53f1531344712d1cf7e2a47693616b764dd4c0a WatchSource:0}: Error finding container 129fa0d0a58b86f9cca8f580e53f1531344712d1cf7e2a47693616b764dd4c0a: Status 404 returned error can't find the container with id 129fa0d0a58b86f9cca8f580e53f1531344712d1cf7e2a47693616b764dd4c0a Sep 09 23:59:59.167813 ip-10-0-128-28 kubenswrapper[2582]: W0909 23:59:59.167793 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25eab51a_ba59_4c67_bf32_89795665a505.slice/crio-88fd1f8e9ce119737cd7b6b3c945483f4bd60a8c33c9f10643699714695f545c WatchSource:0}: Error finding container 88fd1f8e9ce119737cd7b6b3c945483f4bd60a8c33c9f10643699714695f545c: Status 404 returned error can't find the container with id 88fd1f8e9ce119737cd7b6b3c945483f4bd60a8c33c9f10643699714695f545c Sep 09 23:59:59.441039 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.440946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 09 23:59:59.441039 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.441002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 09 23:59:59.441258 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.441100 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:59:59.441258 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.441126 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:59:59.441258 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.441171 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:00.441155281 +0000 UTC m=+3.122088006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:59:59.441258 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.441190 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:00:00.441180803 +0000 UTC m=+3.122113527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:59:59.542160 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.542123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 09 23:59:59.542354 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.542274 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:59:59.542354 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.542298 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:59:59.542354 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.542311 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:59:59.542695 ip-10-0-128-28 kubenswrapper[2582]: E0909 23:59:59.542381 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:00.542361426 +0000 UTC m=+3.223294163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:59:59.768218 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.768070 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 23:54:59 +0000 UTC" deadline="2028-03-12 09:21:59.867670631 +0000 UTC" Sep 09 23:59:59.768218 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.768135 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13185h22m0.099539731s" Sep 09 23:59:59.841258 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.841195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerStarted","Data":"a5fa1a8da9e7aff8a3bdfa9827f7e68c128487d9650ad76ab86e7f61c05f2bbc"} Sep 09 23:59:59.865509 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.865465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" event={"ID":"114ee629-8d9c-490b-a6a0-1e4780b4d7a6","Type":"ContainerStarted","Data":"fc82c4e1a710e2550869d81227d27fbfdcc8f1b653c7cbed4a3c9406ac637484"} Sep 09 23:59:59.877123 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.877087 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" event={"ID":"d5f342fa-ab62-494b-aba9-00bcfa4e6551","Type":"ContainerStarted","Data":"8e8efca0dda9b61625678d768f6a08673be1e163bade2142bf5f1bee8e27da3c"} Sep 09 23:59:59.879727 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.879577 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dcbj" event={"ID":"3ebfc1af-7996-405e-a28c-740cff312cdb","Type":"ContainerStarted","Data":"290a61a625e15f45d9986e22b594662711a19d2a9d1fd4f81504f25b9f8507bd"} Sep 09 23:59:59.897677 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.897639 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" event={"ID":"1a426b2798cb1a9b52d39c4792ff821c","Type":"ContainerStarted","Data":"d951f9ef34c291e0a85f8015f20e564be9d7bf13ba7ae5842c81203aaf01763d"} Sep 09 23:59:59.923102 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.923055 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t8h24" event={"ID":"25eab51a-ba59-4c67-bf32-89795665a505","Type":"ContainerStarted","Data":"88fd1f8e9ce119737cd7b6b3c945483f4bd60a8c33c9f10643699714695f545c"} Sep 09 23:59:59.935075 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.935029 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s8dlv" event={"ID":"69c0d046-adc2-4166-9ce8-f2fd42ff33e8","Type":"ContainerStarted","Data":"129fa0d0a58b86f9cca8f580e53f1531344712d1cf7e2a47693616b764dd4c0a"} Sep 09 23:59:59.951835 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.951791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5g9z9" event={"ID":"dd022f5b-cc5a-409f-89f9-6fe1d42e5748","Type":"ContainerStarted","Data":"1d65d316cecada125739dd086db019dd507d504a82b11b99d942334a8e2b476d"} Sep 09 23:59:59.974942 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.974899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p977b" event={"ID":"f1749c27-f505-440c-924a-9952ec646773","Type":"ContainerStarted","Data":"ea85a8e5305fff103199942fc9c21337bf03877a138a55e76ff44c4375aa7fc9"} Sep 09 23:59:59.985725 ip-10-0-128-28 kubenswrapper[2582]: I0909 23:59:59.985686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"2ea07fbe5ca0fd94c365eddb49a3c46cfb636deddfa2c8a35f3335401da61893"} Sep 10 00:00:00.004752 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.004608 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" event={"ID":"d2d8773881ab781b00886710f6952dd2","Type":"ContainerStarted","Data":"74cc99ccf032232fedb5e72110819bf07f94d43d88908d9e4e03d4ebf65d131b"} Sep 10 00:00:00.159732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.159617 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:00:00.449984 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.449895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:00.449984 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.449956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:00.450386 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.450102 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:00.450386 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.450168 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:00:02.450147723 +0000 UTC m=+5.131080463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:00.450661 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.450631 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:00.450774 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.450694 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:02.450675014 +0000 UTC m=+5.131607739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:00.525666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.522051 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:00:00.551086 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.551049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:00.551286 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.551227 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:00:00.551286 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.551254 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:00:00.551286 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.551267 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:00.551462 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.551332 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:02.551313276 +0000 UTC m=+5.232246027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:00.769309 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.769178 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 23:54:59 +0000 UTC" deadline="2028-05-07 20:19:20.022411197 +0000 UTC" Sep 10 00:00:00.769309 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.769249 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14540h19m19.253167181s" Sep 10 00:00:00.821947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.821915 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:00.822114 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.822062 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:00.822522 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.822497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:00.822647 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.822624 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:00.822727 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:00.822703 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:00.823214 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:00.822798 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:02.470532 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.470473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:02.471531 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.470537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:02.471531 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.470739 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:02.471531 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.470813 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:00:06.470791263 +0000 UTC m=+9.151724002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:02.471830 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.471576 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:02.471830 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.471654 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:06.471636813 +0000 UTC m=+9.152569555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:02.582136 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.582096 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:02.582318 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.582256 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:00:02.582318 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.582274 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:00:02.582318 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.582284 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:02.582523 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.582334 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:06.582320931 +0000 UTC m=+9.263253652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:02.822446 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.822353 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:02.822669 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.822516 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:02.822969 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.822945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:02.823084 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.823061 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:02.823142 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:02.823117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:02.823242 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:02.823188 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:04.822392 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:04.822290 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:04.823433 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:04.822450 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:04.823433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:04.822910 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:04.823433 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:04.823012 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:04.823433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:04.823070 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:04.823433 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:04.823136 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:06.515590 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.515530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:06.516109 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.515606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:06.516109 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.515718 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:06.516109 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.515802 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:14.515780002 +0000 UTC m=+17.196712729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:06.516109 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.515718 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:06.516109 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.515869 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:00:14.515855725 +0000 UTC m=+17.196788463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:06.616776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.616682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:06.616952 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.616896 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:00:06.616952 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.616925 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:00:06.616952 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.616939 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:06.617123 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.617015 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:14.616994283 +0000 UTC m=+17.297927009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:06.821887 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.821837 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:06.822082 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.822000 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:06.822427 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.822407 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:06.822545 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.822511 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:06.822633 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:06.822558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:06.822699 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:06.822655 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:08.822611 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:08.822565 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:08.823094 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:08.822619 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:08.823094 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:08.822573 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:08.823094 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:08.822716 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:08.823094 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:08.822797 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:08.823094 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:08.822889 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:10.821643 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:10.821604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:10.821643 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:10.821639 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:10.822152 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:10.821604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:10.822152 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:10.821767 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:10.822152 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:10.821887 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:10.822152 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:10.821977 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:12.822776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:12.822586 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:12.823202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:12.822586 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:12.823202 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:12.822866 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:12.823202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:12.822586 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:12.823202 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:12.822948 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:12.823202 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:12.823003 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:14.580545 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.580496 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:14.580999 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.580552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:14.580999 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.580685 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:14.580999 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.580761 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:30.580740906 +0000 UTC m=+33.261673648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:14.580999 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.580687 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:14.580999 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.580830 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:00:30.580812729 +0000 UTC m=+33.261745452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:14.681164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.681123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:14.681344 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.681335 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:00:14.681395 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.681352 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:00:14.681395 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.681362 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:14.681503 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.681416 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:30.681399756 +0000 UTC m=+33.362332490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:14.822470 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.822436 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:14.822642 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.822501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:14.822642 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:14.822598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:14.822729 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.822623 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:14.822729 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.822671 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:14.822799 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:14.822749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:16.822282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:16.822227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:16.822282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:16.822276 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:16.822809 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:16.822338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:16.822809 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:16.822358 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:16.822809 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:16.822443 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:16.822809 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:16.822559 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:18.821777 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:18.821743 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:18.821777 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:18.821779 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:18.822210 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:18.821780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:18.822210 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:18.821858 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:18.822210 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:18.821957 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:18.822210 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:18.822041 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:19.072033 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.071877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" event={"ID":"1a426b2798cb1a9b52d39c4792ff821c","Type":"ContainerStarted","Data":"cf971b32799800abb676c3bc579ad0f9118a60d1854c5ded9e52ed1c40721d81"} Sep 10 00:00:19.077363 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.077301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s8dlv" event={"ID":"69c0d046-adc2-4166-9ce8-f2fd42ff33e8","Type":"ContainerStarted","Data":"562df10551f27752fe312c3bbab4f97bdea4f9b67a1217070ba989d9dda1585a"} Sep 10 00:00:19.084312 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.084211 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"8e4edc5dfa77303b313ad45f0df36d7cd1fda6f343423527256ed0d21b5e9f2c"} Sep 10 00:00:19.088382 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.088330 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-28.ec2.internal" podStartSLOduration=20.088315549 podStartE2EDuration="20.088315549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:19.087490448 +0000 UTC m=+21.768423206" watchObservedRunningTime="2026-09-10 00:00:19.088315549 +0000 UTC m=+21.769248291" Sep 10 00:00:19.089088 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.089058 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" event={"ID":"114ee629-8d9c-490b-a6a0-1e4780b4d7a6","Type":"ContainerStarted","Data":"a6e4517a07621b4295dbbe08267a9495335adf9b3d8f47690d4d675ebea39cd0"} Sep 10 00:00:19.106960 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.106905 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s8dlv" podStartSLOduration=0.412296096 podStartE2EDuration="20.106889519s" totalImagesPullingTime="19.694593423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:19.106474587 +0000 UTC m=+21.787407340" watchObservedRunningTime="2026-09-10 00:00:19.106889519 +0000 UTC m=+21.787822259" Sep 10 00:00:19.123652 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:19.123550 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5xbj4" podStartSLOduration=0.59310792 podStartE2EDuration="20.123477985s" totalImagesPullingTime="19.530370065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:19.121800094 +0000 UTC m=+21.802732856" watchObservedRunningTime="2026-09-10 00:00:19.123477985 +0000 UTC m=+21.804410727" Sep 10 00:00:20.092992 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.092733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5g9z9" event={"ID":"dd022f5b-cc5a-409f-89f9-6fe1d42e5748","Type":"ContainerStarted","Data":"60b3be84fc33eb7e1e24569d8e3922d7347f2f0a8e5dfd547e48b198471daeea"} Sep 10 00:00:20.094598 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.094567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p977b" event={"ID":"f1749c27-f505-440c-924a-9952ec646773","Type":"ContainerStarted","Data":"59faefbd6f77372b1c334ee42f1f1eeddff341256f97cadd3f716625b14caa73"} Sep 10 00:00:20.098868 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.098842 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"edc06f2b95ad7992c1cb6a52836d11e423a72f2288a4291f3c64a79309bf52de"} Sep 10 00:00:20.098999 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.098921 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"f2caf5aeee1b4f4c7c70bb5c9dd2b922812ee9c005f673174852606c601ecef8"} Sep 10 00:00:20.098999 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.098964 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"ca4ac1efbffc05e0f65d546c782a088cbe790e645bba44cadc294bd88b7928f7"} Sep 10 00:00:20.099102 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.099008 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"394233c4d9e4118327f32e6e78775f98b0df2422540ce3debf18b802898ef348"} Sep 10 00:00:20.099102 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.099045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"a0baf3318d943c9bf05daf1c28983ec1fa004f384cd6cba810bee35be4cc298e"} Sep 10 00:00:20.100580 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.100545 2582 generic.go:356] "Generic (PLEG): container finished" podID="d2d8773881ab781b00886710f6952dd2" containerID="53479fb8a8570b5b1197ca1f2bc839049842eea59bd6ced1c4ca7eb283ab19e7" exitCode=0 Sep 10 00:00:20.100700 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.100625 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" event={"ID":"d2d8773881ab781b00886710f6952dd2","Type":"ContainerDied","Data":"53479fb8a8570b5b1197ca1f2bc839049842eea59bd6ced1c4ca7eb283ab19e7"} Sep 10 00:00:20.102564 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.102540 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="50c92be0de0bb69a9581189fa3b2f5e52162344e3bdb9c7110c85d601ed59a73" exitCode=0 Sep 10 00:00:20.102675 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.102647 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"50c92be0de0bb69a9581189fa3b2f5e52162344e3bdb9c7110c85d601ed59a73"} Sep 10 00:00:20.104663 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.104637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" event={"ID":"d5f342fa-ab62-494b-aba9-00bcfa4e6551","Type":"ContainerStarted","Data":"f08689c15f91e440790d47f01c1c9c419591caaba8bbd0f32621cee44f797c61"} Sep 10 00:00:20.105834 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.105571 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5g9z9" podStartSLOduration=2.574117521 podStartE2EDuration="22.105553884s" totalImagesPullingTime="19.531436363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:20.10548758 +0000 UTC m=+22.786420336" watchObservedRunningTime="2026-09-10 00:00:20.105553884 +0000 UTC m=+22.786486628" Sep 10 00:00:20.106475 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.106452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dcbj" event={"ID":"3ebfc1af-7996-405e-a28c-740cff312cdb","Type":"ContainerStarted","Data":"4d5aac7690e899b128282b75ea530cfc1739940e8eba70c305b8944135a9bc22"} Sep 10 00:00:20.108926 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.108897 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t8h24" event={"ID":"25eab51a-ba59-4c67-bf32-89795665a505","Type":"ContainerStarted","Data":"7081ab5b7491ee527e4235cfb7d0eed4525bfc6663758e5979b17bdabb67dafd"} Sep 10 00:00:20.151976 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.151912 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p977b" podStartSLOduration=2.596788469 podStartE2EDuration="22.151878206s" totalImagesPullingTime="19.555089737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:20.15137789 +0000 UTC m=+22.832310655" watchObservedRunningTime="2026-09-10 00:00:20.151878206 +0000 UTC m=+22.832810949" Sep 10 00:00:20.180648 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.180477 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4dcbj" podStartSLOduration=2.591375331 podStartE2EDuration="22.180460339s" totalImagesPullingTime="19.589085008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:20.164036377 +0000 UTC m=+22.844969144" watchObservedRunningTime="2026-09-10 00:00:20.180460339 +0000 UTC m=+22.861393072" Sep 10 00:00:20.181191 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.181158 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t8h24" podStartSLOduration=1.6911808320000001 podStartE2EDuration="21.181146972s" totalImagesPullingTime="19.48996614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:20.18064592 +0000 UTC m=+22.861578661" watchObservedRunningTime="2026-09-10 00:00:20.181146972 +0000 UTC m=+22.862079729" Sep 10 00:00:20.822687 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.822657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:20.822855 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.822657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:20.822855 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.822797 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:20.822855 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:20.822795 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:20.823045 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:20.822902 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:20.823045 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:20.823002 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:20.837315 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:20.837287 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 00:00:21.112595 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.112560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" event={"ID":"d2d8773881ab781b00886710f6952dd2","Type":"ContainerStarted","Data":"7bf889bf993cce99398f57f39fea8cedce3683669368ecc8a855e3db03565b39"} Sep 10 00:00:21.115034 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.115001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" event={"ID":"d5f342fa-ab62-494b-aba9-00bcfa4e6551","Type":"ContainerStarted","Data":"c54f1b161f800000ba4372f6170e17227114d465b63fcde1ebf3eaf2e645045b"} Sep 10 00:00:21.125872 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.125813 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-28.ec2.internal" podStartSLOduration=22.125792265 podStartE2EDuration="22.125792265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:21.125360973 +0000 UTC m=+23.806293743" watchObservedRunningTime="2026-09-10 00:00:21.125792265 +0000 UTC m=+23.806725007" Sep 10 00:00:21.796082 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.795944 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T00:00:20.837310936Z","UUID":"19dc3013-45de-41f8-b369-5278f6d0a206","Handler":null,"Name":"","Endpoint":""} Sep 10 00:00:21.797756 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.797702 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 00:00:21.797756 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:21.797754 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 00:00:22.120535 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.120480 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"a36a05efbcbabb7daa43d0bcd3afda6609d4a94bc20c34d091e2416af9f5f7f3"} Sep 10 00:00:22.504196 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.504156 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t8h24" Sep 10 00:00:22.504904 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.504879 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t8h24" Sep 10 00:00:22.822668 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.822581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:22.822831 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.822581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:22.822831 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:22.822711 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:22.822831 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:22.822581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:22.822831 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:22.822820 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:22.822990 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:22.822880 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:23.125902 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:23.125810 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" event={"ID":"d5f342fa-ab62-494b-aba9-00bcfa4e6551","Type":"ContainerStarted","Data":"96e16977bf7704f5d2f3449e98899c79744d50308cb2dbe431f71ec10d4b3b42"} Sep 10 00:00:23.126307 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:23.126155 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t8h24" Sep 10 00:00:23.126756 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:23.126725 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t8h24" Sep 10 00:00:23.141139 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:23.141087 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrspd" podStartSLOduration=2.771288885 podStartE2EDuration="25.141070268s" totalImagesPullingTime="22.369781383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:23.140685681 +0000 UTC m=+25.821618450" watchObservedRunningTime="2026-09-10 00:00:23.141070268 +0000 UTC m=+25.822003010" Sep 10 00:00:24.821721 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:24.821682 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:24.821721 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:24.821713 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:24.822211 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:24.821793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:24.822211 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:24.821807 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:24.822211 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:24.821928 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:24.822211 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:24.822021 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:26.136176 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.135939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" event={"ID":"4dd25809-0f6c-4ff3-86af-c634bbea8253","Type":"ContainerStarted","Data":"47b29f5d583aca110af83938c896220b0554ab3e4a7e383ad08d4f53c6bfacf0"} Sep 10 00:00:26.136978 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.136230 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:00:26.136978 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.136302 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:00:26.137863 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.137835 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="20c96c3514a832abfafea4eb68e3d5cd89341cae838a4ea04df224e1c96b486d" exitCode=0 Sep 10 00:00:26.137965 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.137888 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"20c96c3514a832abfafea4eb68e3d5cd89341cae838a4ea04df224e1c96b486d"} Sep 10 00:00:26.151969 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.151937 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:00:26.162227 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.162157 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" podStartSLOduration=8.333723438 podStartE2EDuration="28.162142207s" totalImagesPullingTime="19.828418769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:26.160918088 +0000 UTC m=+28.841850830" watchObservedRunningTime="2026-09-10 00:00:26.162142207 +0000 UTC m=+28.843074948" Sep 10 00:00:26.821693 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.821666 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:26.821866 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.821660 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:26.821866 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:26.821800 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:26.821866 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:26.821804 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:26.822024 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:26.821873 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:26.822024 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:26.821950 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:27.140480 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.140264 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:00:27.156994 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.156965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:00:27.299590 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.299561 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fc4mk"] Sep 10 00:00:27.299781 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.299669 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:27.299781 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:27.299759 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:27.302617 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.302582 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bxlwt"] Sep 10 00:00:27.302718 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.302698 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:27.302824 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:27.302800 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:27.312798 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.312777 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cqgr"] Sep 10 00:00:27.312890 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:27.312878 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:27.312998 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:27.312975 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:28.143769 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:28.143689 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="8a275cea211009d6a551dccdc1dd3f2cc926574e62e99a31fe08455405b4ff94" exitCode=0 Sep 10 00:00:28.144072 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:28.143773 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"8a275cea211009d6a551dccdc1dd3f2cc926574e62e99a31fe08455405b4ff94"} Sep 10 00:00:28.822342 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:28.822304 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:28.822342 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:28.822335 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:28.822590 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:28.822337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:28.822590 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:28.822433 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:28.822590 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:28.822478 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:28.822590 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:28.822581 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:30.151305 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.151221 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="0ae3971d6a8dd873b13c1caa5d151a0415b3de8f2d6d66629a3c5a4070f472b2" exitCode=0 Sep 10 00:00:30.151305 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.151265 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"0ae3971d6a8dd873b13c1caa5d151a0415b3de8f2d6d66629a3c5a4070f472b2"} Sep 10 00:00:30.600759 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.600718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:30.600759 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.600760 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:30.601012 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.600862 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:30.601012 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.600870 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:30.601012 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.600913 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:01:02.600899712 +0000 UTC m=+65.281832432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:00:30.601012 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.600926 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret podName:5068c29a-75f0-4876-b3cb-7de73ef69a18 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:02.600919918 +0000 UTC m=+65.281852638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret") pod "global-pull-secret-syncer-fc4mk" (UID: "5068c29a-75f0-4876-b3cb-7de73ef69a18") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:00:30.702115 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.702079 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:30.702252 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.702210 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:00:30.702252 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.702225 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:00:30.702252 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.702234 2582 projected.go:196] Error preparing data for projected volume kube-api-access-zstj9 for pod openshift-network-diagnostics/network-check-target-7cqgr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:30.702336 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.702283 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9 podName:5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:02.702269641 +0000 UTC m=+65.383202361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zstj9" (UniqueName: "kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9") pod "network-check-target-7cqgr" (UID: "5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:00:30.821875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.821841 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:30.822013 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.821897 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:30.822013 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:30.821949 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:30.822013 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.821966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc4mk" podUID="5068c29a-75f0-4876-b3cb-7de73ef69a18" Sep 10 00:00:30.822143 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.822029 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:00:30.822143 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:30.822124 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cqgr" podUID="5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11" Sep 10 00:00:31.616780 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.616711 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-28.ec2.internal" event="NodeReady" Sep 10 00:00:31.617190 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.616823 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 00:00:31.646616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.646588 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29816640-2r965"] Sep 10 00:00:31.729618 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.729577 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76fff784fd-nttpn"] Sep 10 00:00:31.729783 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.729700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:31.731843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.731822 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Sep 10 00:00:31.731983 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.731876 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-wq849\"" Sep 10 00:00:31.751632 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.751610 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29816640-2r965"] Sep 10 00:00:31.751742 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.751640 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-smmrm"] Sep 10 00:00:31.751742 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.751716 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.754022 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.754006 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 00:00:31.754305 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.754291 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 00:00:31.754365 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.754332 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5nl5z\"" Sep 10 00:00:31.754877 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.754858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 00:00:31.759078 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.759056 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 00:00:31.777707 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.777683 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fgc5x"] Sep 10 00:00:31.777802 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.777786 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:31.780098 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.780081 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 00:00:31.780098 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.780082 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 00:00:31.780232 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.780166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cvsfg\"" Sep 10 00:00:31.780232 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.780174 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 00:00:31.791851 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.791826 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t8h24_25eab51a-ba59-4c67-bf32-89795665a505/konnectivity-agent/0.log" limit=1024 Sep 10 00:00:31.801794 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.801772 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:31.801794 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.801784 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c7r5j"] Sep 10 00:00:31.805256 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.804181 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gg685\"" Sep 10 00:00:31.805256 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.804407 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 00:00:31.805256 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.804639 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 00:00:31.825945 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.825928 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:31.828557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.828540 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 00:00:31.828666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.828557 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f86ng\"" Sep 10 00:00:31.828666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.828609 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 00:00:31.829094 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.829068 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ngrs"] Sep 10 00:00:31.845949 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.845926 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76fff784fd-nttpn"] Sep 10 00:00:31.845949 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.845953 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-smmrm"] Sep 10 00:00:31.846081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.845967 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fgc5x"] Sep 10 00:00:31.846081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.845980 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ngrs"] Sep 10 00:00:31.846081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.845992 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c7r5j"] Sep 10 00:00:31.846081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.846024 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" Sep 10 00:00:31.848193 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.848168 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 00:00:31.848193 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.848186 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 00:00:31.848350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.848171 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pgwjv\"" Sep 10 00:00:31.912707 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.912707 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912676 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.912707 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnn6x\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc65a25a-30c0-424a-a577-4bc1f09d73ea-cert\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912804 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912828 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:31.912907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.913154 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b03b6f71-f814-4d17-bcd7-f73de5046adf-nginx-conf\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:31.913154 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912944 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.913154 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4d8l\" (UniqueName: \"kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:31.913154 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.912990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljtx\" (UniqueName: \"kubernetes.io/projected/dc65a25a-30c0-424a-a577-4bc1f09d73ea-kube-api-access-rljtx\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:31.913154 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.913088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:31.955689 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:31.955641 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t8h24_25eab51a-ba59-4c67-bf32-89795665a505/konnectivity-agent/0.log" limit=1024 Sep 10 00:00:32.013610 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc65a25a-30c0-424a-a577-4bc1f09d73ea-cert\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:32.013610 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013613 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:32.013610 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nwt\" (UniqueName: \"kubernetes.io/projected/020e04b7-003c-4a26-b8c3-7d3b6ac5b83b-kube-api-access-42nwt\") pod \"network-check-source-685456655f-6ngrs\" (UID: \"020e04b7-003c-4a26-b8c3-7d3b6ac5b83b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e4505d-29bf-4961-9e60-0e37f6b40d54-config-volume\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.013743 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013767 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b03b6f71-f814-4d17-bcd7-f73de5046adf-nginx-conf\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.013821 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:00:32.513802054 +0000 UTC m=+35.194734785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.013910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4d8l\" (UniqueName: \"kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.013942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljtx\" (UniqueName: \"kubernetes.io/projected/dc65a25a-30c0-424a-a577-4bc1f09d73ea-kube-api-access-rljtx\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014028 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e4505d-29bf-4961-9e60-0e37f6b40d54-tmp-dir\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014116 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x4mq\" (UniqueName: \"kubernetes.io/projected/d5e4505d-29bf-4961-9e60-0e37f6b40d54-kube-api-access-5x4mq\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.014418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014216 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnn6x\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.014776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.014467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b03b6f71-f814-4d17-bcd7-f73de5046adf-nginx-conf\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:32.014776 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.014762 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:32.014776 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.014778 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:32.014914 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.014825 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:32.514808194 +0000 UTC m=+35.195740915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:32.015214 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.015174 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.015377 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.015356 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.015792 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.015764 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.015989 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.015967 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:32.018861 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.018714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.018861 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.018738 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc65a25a-30c0-424a-a577-4bc1f09d73ea-cert\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:32.019003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.018735 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.022632 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.022600 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljtx\" (UniqueName: \"kubernetes.io/projected/dc65a25a-30c0-424a-a577-4bc1f09d73ea-kube-api-access-rljtx\") pod \"ingress-canary-smmrm\" (UID: \"dc65a25a-30c0-424a-a577-4bc1f09d73ea\") " pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:32.022855 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.022837 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnn6x\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.023697 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.023673 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.023965 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.023930 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4d8l\" (UniqueName: \"kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l\") pod \"image-pruner-29816640-2r965\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:32.038606 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.038583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:00:32.086095 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.086069 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-smmrm" Sep 10 00:00:32.115466 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e4505d-29bf-4961-9e60-0e37f6b40d54-tmp-dir\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.115591 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x4mq\" (UniqueName: \"kubernetes.io/projected/d5e4505d-29bf-4961-9e60-0e37f6b40d54-kube-api-access-5x4mq\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.115631 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115584 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nwt\" (UniqueName: \"kubernetes.io/projected/020e04b7-003c-4a26-b8c3-7d3b6ac5b83b-kube-api-access-42nwt\") pod \"network-check-source-685456655f-6ngrs\" (UID: \"020e04b7-003c-4a26-b8c3-7d3b6ac5b83b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" Sep 10 00:00:32.115666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e4505d-29bf-4961-9e60-0e37f6b40d54-config-volume\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.115778 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.115778 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.115744 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e4505d-29bf-4961-9e60-0e37f6b40d54-tmp-dir\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.115980 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.115891 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:32.115980 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.115947 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:32.615933359 +0000 UTC m=+35.296866079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:32.116164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.116145 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e4505d-29bf-4961-9e60-0e37f6b40d54-config-volume\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.126224 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.126198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x4mq\" (UniqueName: \"kubernetes.io/projected/d5e4505d-29bf-4961-9e60-0e37f6b40d54-kube-api-access-5x4mq\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.126676 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.126658 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nwt\" (UniqueName: \"kubernetes.io/projected/020e04b7-003c-4a26-b8c3-7d3b6ac5b83b-kube-api-access-42nwt\") pod \"network-check-source-685456655f-6ngrs\" (UID: \"020e04b7-003c-4a26-b8c3-7d3b6ac5b83b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" Sep 10 00:00:32.153899 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.153565 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" Sep 10 00:00:32.360531 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.360476 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29816640-2r965"] Sep 10 00:00:32.365304 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:00:32.365272 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bbd9ae_02c6_43bb_bf1e_8afaaceb980e.slice/crio-493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66 WatchSource:0}: Error finding container 493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66: Status 404 returned error can't find the container with id 493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66 Sep 10 00:00:32.375954 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.375929 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-smmrm"] Sep 10 00:00:32.379575 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:00:32.379553 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc65a25a_30c0_424a_a577_4bc1f09d73ea.slice/crio-397c7aaab77d721623fc2b4aad8b64d6d660ebe9a696e15abeeb2143b12fb9c1 WatchSource:0}: Error finding container 397c7aaab77d721623fc2b4aad8b64d6d660ebe9a696e15abeeb2143b12fb9c1: Status 404 returned error can't find the container with id 397c7aaab77d721623fc2b4aad8b64d6d660ebe9a696e15abeeb2143b12fb9c1 Sep 10 00:00:32.452746 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.452711 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ngrs"] Sep 10 00:00:32.456361 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:00:32.456332 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020e04b7_003c_4a26_b8c3_7d3b6ac5b83b.slice/crio-68733270c88e16df86f2c19d8707a3cbfc87aaf5ff6158328d2a3e49c77ac76d WatchSource:0}: Error finding container 68733270c88e16df86f2c19d8707a3cbfc87aaf5ff6158328d2a3e49c77ac76d: Status 404 returned error can't find the container with id 68733270c88e16df86f2c19d8707a3cbfc87aaf5ff6158328d2a3e49c77ac76d Sep 10 00:00:32.519581 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.519549 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:32.519750 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.519603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:32.519750 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.519697 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:32.519750 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.519716 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:32.519916 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.519760 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:32.519916 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.519770 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:33.519752201 +0000 UTC m=+36.200684943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:32.519916 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.519810 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:00:33.519794321 +0000 UTC m=+36.200727062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:32.620882 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.620844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:32.621602 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.620965 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:32.621602 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:32.621015 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:33.621002478 +0000 UTC m=+36.301935197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:32.822018 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.821979 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:00:32.822205 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.822146 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:00:32.822299 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.822281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:00:32.824872 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.824850 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 00:00:32.824872 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.824870 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 00:00:32.825059 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.824869 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bf5p9\"" Sep 10 00:00:32.825059 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:32.824848 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s66cg\"" Sep 10 00:00:33.160562 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.160303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-smmrm" event={"ID":"dc65a25a-30c0-424a-a577-4bc1f09d73ea","Type":"ContainerStarted","Data":"397c7aaab77d721623fc2b4aad8b64d6d660ebe9a696e15abeeb2143b12fb9c1"} Sep 10 00:00:33.162175 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.162146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29816640-2r965" event={"ID":"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e","Type":"ContainerStarted","Data":"85bd5e86bc5caab40fea72dcc7e6beb4206a35c0d60dcb46d8e35b02530db187"} Sep 10 00:00:33.162309 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.162198 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29816640-2r965" event={"ID":"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e","Type":"ContainerStarted","Data":"493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66"} Sep 10 00:00:33.163256 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.163229 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" event={"ID":"020e04b7-003c-4a26-b8c3-7d3b6ac5b83b","Type":"ContainerStarted","Data":"68733270c88e16df86f2c19d8707a3cbfc87aaf5ff6158328d2a3e49c77ac76d"} Sep 10 00:00:33.176327 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.176274 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-pruner-29816640-2r965" podStartSLOduration=33.176260088 podStartE2EDuration="33.176260088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:33.175021 +0000 UTC m=+35.855953769" watchObservedRunningTime="2026-09-10 00:00:33.176260088 +0000 UTC m=+35.857192829" Sep 10 00:00:33.527697 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.527648 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:33.527888 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.527719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:33.527888 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.527842 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:33.527888 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.527871 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:33.528059 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.527938 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:35.527917293 +0000 UTC m=+38.208850039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:33.528059 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.527950 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:33.528059 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.528005 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:00:35.527989393 +0000 UTC m=+38.208922142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:33.628938 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:33.628905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:33.629390 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.629138 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:33.629390 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:33.629202 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:35.629183438 +0000 UTC m=+38.310116181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:35.544061 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:35.544013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:35.544074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.544181 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.544206 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.544210 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.544266 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:39.544246883 +0000 UTC m=+42.225179630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:35.544593 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.544281 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:00:39.544274517 +0000 UTC m=+42.225207236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:35.645158 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:35.645125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:35.645325 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.645261 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:35.645389 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:35.645330 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:39.645310182 +0000 UTC m=+42.326242919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:36.898870 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:36.898815 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-t8h24_25eab51a-ba59-4c67-bf32-89795665a505/konnectivity-agent/0.log" limit=1024 Sep 10 00:00:39.183316 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:39.183249 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-smmrm" event={"ID":"dc65a25a-30c0-424a-a577-4bc1f09d73ea","Type":"ContainerStarted","Data":"6a30849abd087f7fb03fff265c1d9836bef628eaff7b51f71928f931f767f36b"} Sep 10 00:00:39.197574 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:39.197500 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-smmrm" podStartSLOduration=1.561820489 podStartE2EDuration="8.197478998s" totalImagesPullingTime="6.635658509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:39.196729357 +0000 UTC m=+41.877662100" watchObservedRunningTime="2026-09-10 00:00:39.197478998 +0000 UTC m=+41.878411741" Sep 10 00:00:39.574339 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:39.574296 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:39.574339 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:39.574341 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:39.574581 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.574435 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:39.574581 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.574442 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:39.574581 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.574463 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:39.574581 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.574488 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:00:47.574474852 +0000 UTC m=+50.255407572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:39.574581 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.574508 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:47.574494206 +0000 UTC m=+50.255426930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:39.675556 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:39.675499 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:39.675691 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.675648 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:39.675725 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:39.675711 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:00:47.675696485 +0000 UTC m=+50.356629211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:40.188371 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:40.188337 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="b6393365663054014b8441e88b23d7166180e7dbc3ed3828967541b2fdaba5dc" exitCode=0 Sep 10 00:00:40.188798 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:40.188422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"b6393365663054014b8441e88b23d7166180e7dbc3ed3828967541b2fdaba5dc"} Sep 10 00:00:40.190136 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:40.190111 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" event={"ID":"020e04b7-003c-4a26-b8c3-7d3b6ac5b83b","Type":"ContainerStarted","Data":"b88faf586a0905d2cf84b790d2e372312451ca8c79378f4c40430b337258b1a2"} Sep 10 00:00:40.225266 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:40.225186 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-6ngrs" podStartSLOduration=32.657356134 podStartE2EDuration="39.225165113s" totalImagesPullingTime="6.567808979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:40.224977969 +0000 UTC m=+42.905910733" watchObservedRunningTime="2026-09-10 00:00:40.225165113 +0000 UTC m=+42.906097857" Sep 10 00:00:41.195621 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:41.195586 2582 generic.go:356] "Generic (PLEG): container finished" podID="40e30186-ac33-4218-9719-62255d7f2414" containerID="086c6af95ef888d7a3723f2bffd4f0e3875313481ec1761705bfef4e2601763a" exitCode=0 Sep 10 00:00:41.195991 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:41.195660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerDied","Data":"086c6af95ef888d7a3723f2bffd4f0e3875313481ec1761705bfef4e2601763a"} Sep 10 00:00:42.201206 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:42.201167 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" event={"ID":"40e30186-ac33-4218-9719-62255d7f2414","Type":"ContainerStarted","Data":"d0cec9953e1c3e5a89299b5b933e8bfd54f2a5480ef8bfb52e00301b0a7bb42e"} Sep 10 00:00:42.220900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:42.220853 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ldwtj" podStartSLOduration=7.1721541460000005 podStartE2EDuration="43.220837813s" totalImagesPullingTime="36.048683667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:00:42.219228181 +0000 UTC m=+44.900160933" watchObservedRunningTime="2026-09-10 00:00:42.220837813 +0000 UTC m=+44.901770555" Sep 10 00:00:47.634717 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:47.634676 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:00:47.634717 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:47.634726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:00:47.635164 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.634831 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:00:47.635164 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.634884 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:01:03.634869233 +0000 UTC m=+66.315801967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:00:47.635164 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.634926 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:00:47.635164 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.634946 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:00:47.635164 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.634992 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:03.634980231 +0000 UTC m=+66.315912951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:00:47.735855 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:47.735822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:00:47.736015 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.735940 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:00:47.736015 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:00:47.735995 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:03.735979586 +0000 UTC m=+66.416912330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:00:59.157297 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:00:59.157270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w8rrn" Sep 10 00:01:02.645675 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.645622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:01:02.645675 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.645678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:01:02.648007 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.647985 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 00:01:02.648085 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.648067 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 00:01:02.656893 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:02.656873 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 00:01:02.656960 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:02.656948 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:02:06.656929637 +0000 UTC m=+129.337862362 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : secret "metrics-daemon-secret" not found Sep 10 00:01:02.659739 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.659710 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5068c29a-75f0-4876-b3cb-7de73ef69a18-original-pull-secret\") pod \"global-pull-secret-syncer-fc4mk\" (UID: \"5068c29a-75f0-4876-b3cb-7de73ef69a18\") " pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:01:02.746903 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.746860 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:01:02.749288 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.749260 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zstj9\" (UniqueName: \"kubernetes.io/projected/5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11-kube-api-access-zstj9\") pod \"network-check-target-7cqgr\" (UID: \"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11\") " pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:01:02.835437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.835403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc4mk" Sep 10 00:01:02.845709 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.845679 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bf5p9\"" Sep 10 00:01:02.854579 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:02.854547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:01:03.104115 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.104080 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fc4mk"] Sep 10 00:01:03.107470 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:01:03.107438 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5068c29a_75f0_4876_b3cb_7de73ef69a18.slice/crio-2f61a455d0396d62249c8e44073dfe05dc6f3ba455899de17f26ef8e43495a89 WatchSource:0}: Error finding container 2f61a455d0396d62249c8e44073dfe05dc6f3ba455899de17f26ef8e43495a89: Status 404 returned error can't find the container with id 2f61a455d0396d62249c8e44073dfe05dc6f3ba455899de17f26ef8e43495a89 Sep 10 00:01:03.126547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.126500 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cqgr"] Sep 10 00:01:03.129824 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:01:03.129797 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfa3dd9_4c67_4b09_bd83_4b8cc9a30e11.slice/crio-a10f57054cc462de0b298a4fc8e9eebbf191edf51416657a0318c77094013d65 WatchSource:0}: Error finding container a10f57054cc462de0b298a4fc8e9eebbf191edf51416657a0318c77094013d65: Status 404 returned error can't find the container with id a10f57054cc462de0b298a4fc8e9eebbf191edf51416657a0318c77094013d65 Sep 10 00:01:03.251819 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.251782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cqgr" event={"ID":"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11","Type":"ContainerStarted","Data":"e970b30b0608d276764a36a5ceff9881ba847a08cfaf4094d3b6a4efc9ef87fe"} Sep 10 00:01:03.252000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.251913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:01:03.252000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.251959 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cqgr" event={"ID":"5bfa3dd9-4c67-4b09-bd83-4b8cc9a30e11","Type":"ContainerStarted","Data":"a10f57054cc462de0b298a4fc8e9eebbf191edf51416657a0318c77094013d65"} Sep 10 00:01:03.252986 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.252965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fc4mk" event={"ID":"5068c29a-75f0-4876-b3cb-7de73ef69a18","Type":"ContainerStarted","Data":"2f61a455d0396d62249c8e44073dfe05dc6f3ba455899de17f26ef8e43495a89"} Sep 10 00:01:03.267014 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.266963 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7cqgr" podStartSLOduration=65.266948953 podStartE2EDuration="1m5.266948953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:01:03.266780377 +0000 UTC m=+65.947713118" watchObservedRunningTime="2026-09-10 00:01:03.266948953 +0000 UTC m=+65.947881691" Sep 10 00:01:03.652135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.652081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:01:03.652135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.652141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:01:03.652686 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.652234 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:01:03.652686 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.652286 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:01:35.652272733 +0000 UTC m=+98.333205452 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:01:03.652686 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.652232 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:01:03.652686 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.652329 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:01:03.652686 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.652382 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:35.652368473 +0000 UTC m=+98.333301207 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:01:03.753174 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:03.753128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:01:03.753339 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.753248 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:01:03.753339 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:03.753299 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:01:35.753286559 +0000 UTC m=+98.434219280 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:01:08.267865 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:08.267822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fc4mk" event={"ID":"5068c29a-75f0-4876-b3cb-7de73ef69a18","Type":"ContainerStarted","Data":"02f7d091746ad89018f471cc9a25bdb7276e64bf873ac511dec90dc263b57eaf"} Sep 10 00:01:08.282341 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:08.282286 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fc4mk" podStartSLOduration=64.636768333 podStartE2EDuration="1m9.282267419s" totalImagesPullingTime="4.645499086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:01:08.282227993 +0000 UTC m=+70.963160755" watchObservedRunningTime="2026-09-10 00:01:08.282267419 +0000 UTC m=+70.963200164" Sep 10 00:01:34.258209 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:34.258178 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7cqgr" Sep 10 00:01:35.693402 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:35.693348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:01:35.693402 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:35.693406 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:01:35.694061 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.693540 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:01:35.694061 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.693595 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:02:39.693580077 +0000 UTC m=+162.374512796 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:01:35.694061 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.693537 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:01:35.694061 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.693613 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:01:35.694061 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.693641 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:02:39.693630971 +0000 UTC m=+162.374563708 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:01:35.794330 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:01:35.794287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:01:35.794481 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.794435 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:01:35.794539 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:01:35.794498 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:02:39.794483778 +0000 UTC m=+162.475416498 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:02:06.718623 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:06.718562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:02:06.719105 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:06.718737 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 00:02:06.719105 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:06.718813 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:04:08.718795601 +0000 UTC m=+251.399728342 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : secret "metrics-daemon-secret" not found Sep 10 00:02:34.760136 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:34.760091 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" podUID="d8f88911-2bce-4790-a5f7-503d7a4956e3" Sep 10 00:02:34.811425 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:34.811387 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" podUID="b03b6f71-f814-4d17-bcd7-f73de5046adf" Sep 10 00:02:34.833828 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:34.833791 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-c7r5j" podUID="d5e4505d-29bf-4961-9e60-0e37f6b40d54" Sep 10 00:02:35.471359 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:35.471326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:02:35.471570 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:35.471326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:02:35.471570 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:35.471326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:02:35.848211 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:35.848172 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:02:39.764039 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:39.764001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:39.764078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.764165 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.764170 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.764244 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:04:41.764227811 +0000 UTC m=+284.445160534 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.764177 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:02:39.764422 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.764299 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:41.764287032 +0000 UTC m=+284.445219758 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:02:39.864745 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:39.864704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:02:39.864908 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.864844 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:02:39.864951 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:02:39.864927 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:41.864908398 +0000 UTC m=+284.545841134 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:02:48.821784 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:02:48.821733 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:04:08.799251 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:08.799202 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:04:08.799759 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:08.799336 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 00:04:08.799759 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:08.799398 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs podName:2cb4843b-acb3-45bd-98c1-11acccb93f3a nodeName:}" failed. No retries permitted until 2026-09-10 00:06:10.799385019 +0000 UTC m=+373.480317752 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs") pod "network-metrics-daemon-bxlwt" (UID: "2cb4843b-acb3-45bd-98c1-11acccb93f3a") : secret "metrics-daemon-secret" not found Sep 10 00:04:30.590035 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.590001 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wvbvd"] Sep 10 00:04:30.592126 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.592105 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.592971 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.592949 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll"] Sep 10 00:04:30.595040 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595013 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 00:04:30.595160 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595046 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vnp77\"" Sep 10 00:04:30.595160 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595064 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq"] Sep 10 00:04:30.595160 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595052 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 00:04:30.595355 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595222 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.595855 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.595837 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 00:04:30.597459 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.596251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 00:04:30.600293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.600068 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.600293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.600123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xspfw\"" Sep 10 00:04:30.600674 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.600653 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 00:04:30.600782 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.600672 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 00:04:30.600782 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.600698 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 00:04:30.601213 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.601155 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 00:04:30.603427 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603411 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 00:04:30.603699 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603680 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 00:04:30.603783 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603760 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 00:04:30.603912 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603890 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 00:04:30.604022 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603893 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-jfgpd\"" Sep 10 00:04:30.604022 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.603894 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 00:04:30.610730 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.610649 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wvbvd"] Sep 10 00:04:30.611750 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.611732 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll"] Sep 10 00:04:30.616317 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.616295 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq"] Sep 10 00:04:30.653685 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8157b04a-9036-4a96-a570-eb8d0e22dba5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.653685 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-serving-cert\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60945d27-560a-4451-8173-ea86bcb8812c-serving-cert\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9jl\" (UniqueName: \"kubernetes.io/projected/60945d27-560a-4451-8173-ea86bcb8812c-kube-api-access-dm9jl\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcg9n\" (UniqueName: \"kubernetes.io/projected/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-kube-api-access-pcg9n\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5znrg\" (UniqueName: \"kubernetes.io/projected/8157b04a-9036-4a96-a570-eb8d0e22dba5-kube-api-access-5znrg\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.653875 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653870 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60945d27-560a-4451-8173-ea86bcb8812c-config\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.654078 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-trusted-ca\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.654078 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.653956 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-config\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.689725 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.689693 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64cc444bc6-mqvdc"] Sep 10 00:04:30.692061 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.692045 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.694179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694156 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 00:04:30.694378 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694355 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 00:04:30.694378 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694371 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 00:04:30.694657 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694636 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 00:04:30.694778 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694696 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-njlqh\"" Sep 10 00:04:30.694854 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694780 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 00:04:30.694919 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.694861 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 00:04:30.705108 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.705087 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64cc444bc6-mqvdc"] Sep 10 00:04:30.754617 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754586 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9jl\" (UniqueName: \"kubernetes.io/projected/60945d27-560a-4451-8173-ea86bcb8812c-kube-api-access-dm9jl\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.754776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754628 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcg9n\" (UniqueName: \"kubernetes.io/projected/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-kube-api-access-pcg9n\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.754776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754656 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.754864 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5znrg\" (UniqueName: \"kubernetes.io/projected/8157b04a-9036-4a96-a570-eb8d0e22dba5-kube-api-access-5znrg\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.754864 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.754864 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60945d27-560a-4451-8173-ea86bcb8812c-config\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.754976 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:30.754873 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:30.754976 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:30.754941 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:31.254920178 +0000 UTC m=+273.935852910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:30.754976 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.754872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5jff\" (UniqueName: \"kubernetes.io/projected/b4df5758-cbe1-44c4-9e0f-defca92e159b-kube-api-access-s5jff\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.755092 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-trusted-ca\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.755092 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-config\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.755092 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755061 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-default-certificate\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.755387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-stats-auth\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.755387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755211 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8157b04a-9036-4a96-a570-eb8d0e22dba5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.755387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-serving-cert\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.755387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60945d27-560a-4451-8173-ea86bcb8812c-serving-cert\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.755387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.755661 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755543 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60945d27-560a-4451-8173-ea86bcb8812c-config\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.755830 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755805 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-config\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.756005 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.755980 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8157b04a-9036-4a96-a570-eb8d0e22dba5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.756078 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.756033 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-trusted-ca\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.757527 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.757495 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60945d27-560a-4451-8173-ea86bcb8812c-serving-cert\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.757619 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.757529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-serving-cert\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.765621 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.765601 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcg9n\" (UniqueName: \"kubernetes.io/projected/ef122cd0-dddf-43ac-9deb-bf8c52a04f8e-kube-api-access-pcg9n\") pod \"console-operator-85b454874b-wvbvd\" (UID: \"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e\") " pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.765948 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.765932 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9jl\" (UniqueName: \"kubernetes.io/projected/60945d27-560a-4451-8173-ea86bcb8812c-kube-api-access-dm9jl\") pod \"service-ca-operator-859f94c4b-2zzfq\" (UID: \"60945d27-560a-4451-8173-ea86bcb8812c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:30.766094 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.766076 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5znrg\" (UniqueName: \"kubernetes.io/projected/8157b04a-9036-4a96-a570-eb8d0e22dba5-kube-api-access-5znrg\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:30.856425 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.856333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-stats-auth\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.856425 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.856405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.856652 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.856438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.856652 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.856469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5jff\" (UniqueName: \"kubernetes.io/projected/b4df5758-cbe1-44c4-9e0f-defca92e159b-kube-api-access-s5jff\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.856652 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:30.856572 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 00:04:30.856652 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.856601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-default-certificate\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.856652 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:30.856642 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:31.356622289 +0000 UTC m=+274.037555030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : secret "router-metrics-certs-default" not found Sep 10 00:04:30.856822 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:30.856662 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:31.356655226 +0000 UTC m=+274.037587947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:30.858916 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.858889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-default-certificate\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.859013 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.858891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-stats-auth\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.866179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.866156 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5jff\" (UniqueName: \"kubernetes.io/projected/b4df5758-cbe1-44c4-9e0f-defca92e159b-kube-api-access-s5jff\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:30.907593 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.907565 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:30.920412 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:30.920387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" Sep 10 00:04:31.175893 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.175814 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wvbvd"] Sep 10 00:04:31.179074 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:04:31.179049 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef122cd0_dddf_43ac_9deb_bf8c52a04f8e.slice/crio-be6c649161dffb76d6c963aed7554e55aebe335f33f61c833f00c6ded57476c1 WatchSource:0}: Error finding container be6c649161dffb76d6c963aed7554e55aebe335f33f61c833f00c6ded57476c1: Status 404 returned error can't find the container with id be6c649161dffb76d6c963aed7554e55aebe335f33f61c833f00c6ded57476c1 Sep 10 00:04:31.212093 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.212062 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq"] Sep 10 00:04:31.214949 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:04:31.214926 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60945d27_560a_4451_8173_ea86bcb8812c.slice/crio-bc98a42db0bca0e09fe66c4d2dc81b8ec5c2ffd644d0d6af1d5b9a9be50a4358 WatchSource:0}: Error finding container bc98a42db0bca0e09fe66c4d2dc81b8ec5c2ffd644d0d6af1d5b9a9be50a4358: Status 404 returned error can't find the container with id bc98a42db0bca0e09fe66c4d2dc81b8ec5c2ffd644d0d6af1d5b9a9be50a4358 Sep 10 00:04:31.259637 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.259588 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:31.259809 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:31.259734 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:31.259809 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:31.259798 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:32.259783454 +0000 UTC m=+274.940716174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:31.361038 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.360987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:31.361224 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.361139 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:31.361224 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:31.361181 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:32.361158375 +0000 UTC m=+275.042091107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:31.361334 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:31.361238 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 00:04:31.361334 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:31.361282 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:32.361269094 +0000 UTC m=+275.042201815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : secret "router-metrics-certs-default" not found Sep 10 00:04:31.751929 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.751891 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" event={"ID":"60945d27-560a-4451-8173-ea86bcb8812c","Type":"ContainerStarted","Data":"bc98a42db0bca0e09fe66c4d2dc81b8ec5c2ffd644d0d6af1d5b9a9be50a4358"} Sep 10 00:04:31.752889 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:31.752867 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" event={"ID":"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e","Type":"ContainerStarted","Data":"be6c649161dffb76d6c963aed7554e55aebe335f33f61c833f00c6ded57476c1"} Sep 10 00:04:32.270531 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:32.270477 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:32.270739 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:32.270657 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:32.270812 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:32.270743 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:34.270716326 +0000 UTC m=+276.951649047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:32.371834 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:32.371663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:32.371834 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:32.371788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:32.371834 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:32.371837 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:34.371815204 +0000 UTC m=+277.052747942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:32.372122 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:32.371899 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 00:04:32.372122 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:32.371968 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:34.371950482 +0000 UTC m=+277.052883216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : secret "router-metrics-certs-default" not found Sep 10 00:04:34.286663 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.286602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:34.287044 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:34.286745 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:34.287044 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:34.286812 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:38.28679696 +0000 UTC m=+280.967729679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:34.387347 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.387299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:34.387508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.387362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:34.387508 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:34.387470 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 00:04:34.387603 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:34.387505 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:38.387481867 +0000 UTC m=+281.068414589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:34.387603 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:34.387557 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:38.387545578 +0000 UTC m=+281.068478305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : secret "router-metrics-certs-default" not found Sep 10 00:04:34.762835 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.762799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" event={"ID":"60945d27-560a-4451-8173-ea86bcb8812c","Type":"ContainerStarted","Data":"0b94977f18e0b158fc017e571087a8ca5ea789fe5791fd77c75e98f2dcc3a86e"} Sep 10 00:04:34.764346 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.764325 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/0.log" Sep 10 00:04:34.764449 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.764360 2582 generic.go:356] "Generic (PLEG): container finished" podID="ef122cd0-dddf-43ac-9deb-bf8c52a04f8e" containerID="f113f88ba58c1f0515f9942addef191e1183a750efa03e7e3e1edf0c61be1e7b" exitCode=255 Sep 10 00:04:34.764449 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.764384 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" event={"ID":"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e","Type":"ContainerDied","Data":"f113f88ba58c1f0515f9942addef191e1183a750efa03e7e3e1edf0c61be1e7b"} Sep 10 00:04:34.764622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.764609 2582 scope.go:117] "RemoveContainer" containerID="f113f88ba58c1f0515f9942addef191e1183a750efa03e7e3e1edf0c61be1e7b" Sep 10 00:04:34.779036 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.778990 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" podStartSLOduration=2.263221329 podStartE2EDuration="4.778973873s" totalImagesPullingTime="2.515752544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:04:34.777580396 +0000 UTC m=+277.458513155" watchObservedRunningTime="2026-09-10 00:04:34.778973873 +0000 UTC m=+277.459906665" Sep 10 00:04:34.966448 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.966413 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zz872"] Sep 10 00:04:34.968628 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.968606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:34.971202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.971177 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 00:04:34.971202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.971193 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 00:04:34.971370 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.971185 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7nf9b\"" Sep 10 00:04:34.971370 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.971185 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 00:04:34.971527 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.971498 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 00:04:34.979302 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:34.979272 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zz872"] Sep 10 00:04:35.093168 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.093070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwjrm\" (UniqueName: \"kubernetes.io/projected/87eede89-4ffe-4c39-9027-6490743cb16d-kube-api-access-jwjrm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.093168 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.093145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87eede89-4ffe-4c39-9027-6490743cb16d-data-volume\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.093350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.093204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.093350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.093235 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87eede89-4ffe-4c39-9027-6490743cb16d-crio-socket\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.093350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.093252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.193829 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.193795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194004 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.193837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87eede89-4ffe-4c39-9027-6490743cb16d-crio-socket\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194004 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.193857 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194004 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.193936 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:04:35.194004 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.193947 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87eede89-4ffe-4c39-9027-6490743cb16d-crio-socket\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194004 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.194003 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:35.693982455 +0000 UTC m=+278.374915182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : configmap "kube-rbac-proxy" not found Sep 10 00:04:35.194208 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.194050 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:04:35.194208 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.194110 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:35.694096953 +0000 UTC m=+278.375029677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : secret "insights-runtime-extractor-tls" not found Sep 10 00:04:35.194208 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.194167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwjrm\" (UniqueName: \"kubernetes.io/projected/87eede89-4ffe-4c39-9027-6490743cb16d-kube-api-access-jwjrm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194208 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.194201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87eede89-4ffe-4c39-9027-6490743cb16d-data-volume\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.194473 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.194459 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87eede89-4ffe-4c39-9027-6490743cb16d-data-volume\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.208318 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.208284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwjrm\" (UniqueName: \"kubernetes.io/projected/87eede89-4ffe-4c39-9027-6490743cb16d-kube-api-access-jwjrm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.698540 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.698476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.699008 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.698639 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:35.699008 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.698646 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:04:35.699008 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.698815 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:36.698799555 +0000 UTC m=+279.379732278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : secret "insights-runtime-extractor-tls" not found Sep 10 00:04:35.699008 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.698679 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:04:35.699008 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.698877 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:36.69886052 +0000 UTC m=+279.379793315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : configmap "kube-rbac-proxy" not found Sep 10 00:04:35.769752 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.769724 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:04:35.770134 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.770118 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/0.log" Sep 10 00:04:35.770187 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.770153 2582 generic.go:356] "Generic (PLEG): container finished" podID="ef122cd0-dddf-43ac-9deb-bf8c52a04f8e" containerID="c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240" exitCode=255 Sep 10 00:04:35.770234 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.770220 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" event={"ID":"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e","Type":"ContainerDied","Data":"c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240"} Sep 10 00:04:35.770305 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.770294 2582 scope.go:117] "RemoveContainer" containerID="f113f88ba58c1f0515f9942addef191e1183a750efa03e7e3e1edf0c61be1e7b" Sep 10 00:04:35.770498 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:35.770477 2582 scope.go:117] "RemoveContainer" containerID="c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240" Sep 10 00:04:35.770677 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:35.770657 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wvbvd_openshift-console-operator(ef122cd0-dddf-43ac-9deb-bf8c52a04f8e)\"" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" podUID="ef122cd0-dddf-43ac-9deb-bf8c52a04f8e" Sep 10 00:04:36.707316 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:36.707280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:36.707316 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:36.707323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:36.707836 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:36.707398 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:04:36.707836 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:36.707428 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:04:36.707836 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:36.707450 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:38.707436765 +0000 UTC m=+281.388369485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : configmap "kube-rbac-proxy" not found Sep 10 00:04:36.707836 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:36.707480 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:38.707467271 +0000 UTC m=+281.388399991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : secret "insights-runtime-extractor-tls" not found Sep 10 00:04:36.774585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:36.774554 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:04:36.774906 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:36.774890 2582 scope.go:117] "RemoveContainer" containerID="c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240" Sep 10 00:04:36.775048 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:36.775031 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wvbvd_openshift-console-operator(ef122cd0-dddf-43ac-9deb-bf8c52a04f8e)\"" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" podUID="ef122cd0-dddf-43ac-9deb-bf8c52a04f8e" Sep 10 00:04:36.935799 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:36.935766 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5g9z9_dd022f5b-cc5a-409f-89f9-6fe1d42e5748/dns-node-resolver/0.log" Sep 10 00:04:37.336163 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:37.336130 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29816640-2r965_41bbd9ae-02c6-43bb-bf1e-8afaaceb980e/image-pruner/0.log" Sep 10 00:04:38.135450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.135419 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p977b_f1749c27-f505-440c-924a-9952ec646773/node-ca/0.log" Sep 10 00:04:38.320000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.319962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:38.320143 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.320117 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:38.320209 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.320198 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:04:46.320182293 +0000 UTC m=+289.001115012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:38.420954 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.420871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:38.420954 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.420937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:38.421140 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.421044 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:46.421025503 +0000 UTC m=+289.101958229 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:38.421140 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.421070 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 00:04:38.421140 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.421116 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:04:46.42110383 +0000 UTC m=+289.102036550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : secret "router-metrics-certs-default" not found Sep 10 00:04:38.472087 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.472053 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" podUID="b03b6f71-f814-4d17-bcd7-f73de5046adf" Sep 10 00:04:38.472087 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.472053 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" podUID="d8f88911-2bce-4790-a5f7-503d7a4956e3" Sep 10 00:04:38.472235 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.472054 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-c7r5j" podUID="d5e4505d-29bf-4961-9e60-0e37f6b40d54" Sep 10 00:04:38.723996 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.723964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:38.724148 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.724015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:38.724148 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.724116 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:04:38.724148 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.724133 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:04:38.724259 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.724181 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:42.724166039 +0000 UTC m=+285.405098764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : secret "insights-runtime-extractor-tls" not found Sep 10 00:04:38.724259 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:38.724196 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:42.724190224 +0000 UTC m=+285.405122943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : configmap "kube-rbac-proxy" not found Sep 10 00:04:38.780412 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.780381 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:04:38.780591 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.780382 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:04:38.780591 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:38.780381 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:04:39.135416 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:39.135329 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-smmrm_dc65a25a-30c0-424a-a577-4bc1f09d73ea/serve-healthcheck-canary/0.log" Sep 10 00:04:40.908035 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:40.907950 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:40.908035 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:40.908003 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:40.908433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:40.908338 2582 scope.go:117] "RemoveContainer" containerID="c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240" Sep 10 00:04:40.908506 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:40.908486 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wvbvd_openshift-console-operator(ef122cd0-dddf-43ac-9deb-bf8c52a04f8e)\"" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" podUID="ef122cd0-dddf-43ac-9deb-bf8c52a04f8e" Sep 10 00:04:41.847016 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:41.846978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") pod \"image-registry-76fff784fd-nttpn\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:04:41.847016 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:41.847016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:04:41.847217 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.847104 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 00:04:41.847217 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.847123 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 00:04:41.847217 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.847141 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fff784fd-nttpn: secret "image-registry-tls" not found Sep 10 00:04:41.847217 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.847155 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert podName:b03b6f71-f814-4d17-bcd7-f73de5046adf nodeName:}" failed. No retries permitted until 2026-09-10 00:06:43.847143071 +0000 UTC m=+406.528075790 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fgc5x" (UID: "b03b6f71-f814-4d17-bcd7-f73de5046adf") : secret "networking-console-plugin-cert" not found Sep 10 00:04:41.847217 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.847198 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls podName:d8f88911-2bce-4790-a5f7-503d7a4956e3 nodeName:}" failed. No retries permitted until 2026-09-10 00:06:43.847182282 +0000 UTC m=+406.528115022 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls") pod "image-registry-76fff784fd-nttpn" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3") : secret "image-registry-tls" not found Sep 10 00:04:41.947768 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:41.947739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:04:41.948198 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.947888 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:04:41.948198 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:41.947975 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls podName:d5e4505d-29bf-4961-9e60-0e37f6b40d54 nodeName:}" failed. No retries permitted until 2026-09-10 00:06:43.94795373 +0000 UTC m=+406.628886453 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls") pod "dns-default-c7r5j" (UID: "d5e4505d-29bf-4961-9e60-0e37f6b40d54") : secret "dns-default-metrics-tls" not found Sep 10 00:04:42.756180 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:42.756140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:42.756377 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:42.756272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:42.756377 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:42.756280 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:04:42.756377 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:42.756355 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:50.756334468 +0000 UTC m=+293.437267204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : configmap "kube-rbac-proxy" not found Sep 10 00:04:42.756377 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:42.756377 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:04:42.756633 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:42.756430 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls podName:87eede89-4ffe-4c39-9027-6490743cb16d nodeName:}" failed. No retries permitted until 2026-09-10 00:04:50.756418757 +0000 UTC m=+293.437351500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zz872" (UID: "87eede89-4ffe-4c39-9027-6490743cb16d") : secret "insights-runtime-extractor-tls" not found Sep 10 00:04:46.389463 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:46.389419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:04:46.389875 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:46.389580 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:46.389875 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:46.389648 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls podName:8157b04a-9036-4a96-a570-eb8d0e22dba5 nodeName:}" failed. No retries permitted until 2026-09-10 00:05:02.389632645 +0000 UTC m=+305.070565366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xtpll" (UID: "8157b04a-9036-4a96-a570-eb8d0e22dba5") : secret "cluster-monitoring-operator-tls" not found Sep 10 00:04:46.490859 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:46.490823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:46.490968 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:46.490901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:46.491002 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:46.490993 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle podName:b4df5758-cbe1-44c4-9e0f-defca92e159b nodeName:}" failed. No retries permitted until 2026-09-10 00:05:02.490974469 +0000 UTC m=+305.171907194 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle") pod "router-default-64cc444bc6-mqvdc" (UID: "b4df5758-cbe1-44c4-9e0f-defca92e159b") : configmap references non-existent config key: service-ca.crt Sep 10 00:04:46.493211 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:46.493183 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4df5758-cbe1-44c4-9e0f-defca92e159b-metrics-certs\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:04:50.827791 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:50.827747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:50.828253 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:50.827808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:50.828373 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:50.828353 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87eede89-4ffe-4c39-9027-6490743cb16d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:50.830067 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:50.830037 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87eede89-4ffe-4c39-9027-6490743cb16d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zz872\" (UID: \"87eede89-4ffe-4c39-9027-6490743cb16d\") " pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:50.877240 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:50.877200 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zz872" Sep 10 00:04:51.146463 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:51.146356 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zz872"] Sep 10 00:04:51.149929 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:04:51.149897 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87eede89_4ffe_4c39_9027_6490743cb16d.slice/crio-9c7388a8e10e554b413a56b95e914a39047a2b36eb8cd40de3c5fea5f47fada7 WatchSource:0}: Error finding container 9c7388a8e10e554b413a56b95e914a39047a2b36eb8cd40de3c5fea5f47fada7: Status 404 returned error can't find the container with id 9c7388a8e10e554b413a56b95e914a39047a2b36eb8cd40de3c5fea5f47fada7 Sep 10 00:04:51.822207 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:04:51.822161 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bxlwt" podUID="2cb4843b-acb3-45bd-98c1-11acccb93f3a" Sep 10 00:04:51.825104 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:51.825075 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zz872" event={"ID":"87eede89-4ffe-4c39-9027-6490743cb16d","Type":"ContainerStarted","Data":"fca76838bb3155c24ee3d7f961b30efc89f84fdb73ba1972860beaa1fbb2d2b6"} Sep 10 00:04:51.825213 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:51.825123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zz872" event={"ID":"87eede89-4ffe-4c39-9027-6490743cb16d","Type":"ContainerStarted","Data":"9c7388a8e10e554b413a56b95e914a39047a2b36eb8cd40de3c5fea5f47fada7"} Sep 10 00:04:53.831092 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:53.831054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zz872" event={"ID":"87eede89-4ffe-4c39-9027-6490743cb16d","Type":"ContainerStarted","Data":"d4d711cb14c2806b563703e647c591feaeec4aad931ca0f4dcdc1187577b6838"} Sep 10 00:04:54.837597 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:54.837561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zz872" event={"ID":"87eede89-4ffe-4c39-9027-6490743cb16d","Type":"ContainerStarted","Data":"91f2a137c99d1d0cddfe2d8f63ebfac406c32a2d9d076c28408214a0d4a2d9d5"} Sep 10 00:04:54.853480 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:54.853434 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zz872" podStartSLOduration=17.504243865 podStartE2EDuration="20.853417485s" totalImagesPullingTime="3.34917362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:04:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:04:54.852431707 +0000 UTC m=+297.533364485" watchObservedRunningTime="2026-09-10 00:04:54.853417485 +0000 UTC m=+297.534350227" Sep 10 00:04:55.822338 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:55.822303 2582 scope.go:117] "RemoveContainer" containerID="c4b945c54f41fe083c853981b1daf94a94a0e9f4ea6e4db041ca56b3ad644240" Sep 10 00:04:56.281164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.281129 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-m6248"] Sep 10 00:04:56.284083 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.284067 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:04:56.286082 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.286056 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 00:04:56.286264 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.286242 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kq8h2\"" Sep 10 00:04:56.286348 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.286248 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 00:04:56.291669 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.291648 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-m6248"] Sep 10 00:04:56.376594 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.376549 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm4n4\" (UniqueName: \"kubernetes.io/projected/2c1ae540-1b98-45a5-93ae-d7eab119984c-kube-api-access-mm4n4\") pod \"downloads-85d98c998f-m6248\" (UID: \"2c1ae540-1b98-45a5-93ae-d7eab119984c\") " pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:04:56.476986 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.476944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm4n4\" (UniqueName: \"kubernetes.io/projected/2c1ae540-1b98-45a5-93ae-d7eab119984c-kube-api-access-mm4n4\") pod \"downloads-85d98c998f-m6248\" (UID: \"2c1ae540-1b98-45a5-93ae-d7eab119984c\") " pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:04:56.484532 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.484483 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm4n4\" (UniqueName: \"kubernetes.io/projected/2c1ae540-1b98-45a5-93ae-d7eab119984c-kube-api-access-mm4n4\") pod \"downloads-85d98c998f-m6248\" (UID: \"2c1ae540-1b98-45a5-93ae-d7eab119984c\") " pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:04:56.592745 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.592657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:04:56.846449 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.846386 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:04:56.846449 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.846434 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" event={"ID":"ef122cd0-dddf-43ac-9deb-bf8c52a04f8e","Type":"ContainerStarted","Data":"61860a49cb1b1ca5e323020f86ea1ad94bf741ef7538c3023f70a792e3bfc425"} Sep 10 00:04:56.846723 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.846698 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:56.851498 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.851476 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" Sep 10 00:04:56.861920 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.861898 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-m6248"] Sep 10 00:04:56.864267 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:56.864227 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-wvbvd" podStartSLOduration=24.319461273 podStartE2EDuration="26.864212995s" totalImagesPullingTime="2.544751722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:04:56.863551246 +0000 UTC m=+299.544484007" watchObservedRunningTime="2026-09-10 00:04:56.864212995 +0000 UTC m=+299.545145737" Sep 10 00:04:56.866151 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:04:56.866130 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1ae540_1b98_45a5_93ae_d7eab119984c.slice/crio-b6fc8c3c1e90e42188406bdae5ab710fcf831e7ac872aa5b6c6d907e239f2a37 WatchSource:0}: Error finding container b6fc8c3c1e90e42188406bdae5ab710fcf831e7ac872aa5b6c6d907e239f2a37: Status 404 returned error can't find the container with id b6fc8c3c1e90e42188406bdae5ab710fcf831e7ac872aa5b6c6d907e239f2a37 Sep 10 00:04:57.727924 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:57.727891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:04:57.728368 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:57.728328 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:04:57.740462 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:57.740440 2582 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 00:04:57.850928 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:04:57.850781 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-m6248" event={"ID":"2c1ae540-1b98-45a5-93ae-d7eab119984c","Type":"ContainerStarted","Data":"b6fc8c3c1e90e42188406bdae5ab710fcf831e7ac872aa5b6c6d907e239f2a37"} Sep 10 00:05:02.433780 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.433743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:05:02.436372 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.436345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8157b04a-9036-4a96-a570-eb8d0e22dba5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xtpll\" (UID: \"8157b04a-9036-4a96-a570-eb8d0e22dba5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:05:02.535369 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.535322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:02.536032 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.536006 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4df5758-cbe1-44c4-9e0f-defca92e159b-service-ca-bundle\") pod \"router-default-64cc444bc6-mqvdc\" (UID: \"b4df5758-cbe1-44c4-9e0f-defca92e159b\") " pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:02.715849 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.715768 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xspfw\"" Sep 10 00:05:02.724528 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.724492 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" Sep 10 00:05:02.803307 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.803279 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-njlqh\"" Sep 10 00:05:02.811910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:02.811883 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:03.004359 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.004329 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll"] Sep 10 00:05:03.013151 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:03.013115 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8157b04a_9036_4a96_a570_eb8d0e22dba5.slice/crio-d680708024000ed4f56f51044d454960e893708e9f83f9a2f14be03f9fda31de WatchSource:0}: Error finding container d680708024000ed4f56f51044d454960e893708e9f83f9a2f14be03f9fda31de: Status 404 returned error can't find the container with id d680708024000ed4f56f51044d454960e893708e9f83f9a2f14be03f9fda31de Sep 10 00:05:03.015596 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.015575 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 00:05:03.089380 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.089346 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64cc444bc6-mqvdc"] Sep 10 00:05:03.092788 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:03.092756 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4df5758_cbe1_44c4_9e0f_defca92e159b.slice/crio-ac8736b14252f8fcd6cd146a86993303c5f141f3fb8801de1579119866cc4d14 WatchSource:0}: Error finding container ac8736b14252f8fcd6cd146a86993303c5f141f3fb8801de1579119866cc4d14: Status 404 returned error can't find the container with id ac8736b14252f8fcd6cd146a86993303c5f141f3fb8801de1579119866cc4d14 Sep 10 00:05:03.874793 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.874748 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" event={"ID":"b4df5758-cbe1-44c4-9e0f-defca92e159b","Type":"ContainerStarted","Data":"3235ee8d5279326d9acdae90b655a078fc1da57c14f49aab99f46f001a91309a"} Sep 10 00:05:03.875252 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.874810 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" event={"ID":"b4df5758-cbe1-44c4-9e0f-defca92e159b","Type":"ContainerStarted","Data":"ac8736b14252f8fcd6cd146a86993303c5f141f3fb8801de1579119866cc4d14"} Sep 10 00:05:03.876039 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.876006 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" event={"ID":"8157b04a-9036-4a96-a570-eb8d0e22dba5","Type":"ContainerStarted","Data":"d680708024000ed4f56f51044d454960e893708e9f83f9a2f14be03f9fda31de"} Sep 10 00:05:03.894297 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:03.894250 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" podStartSLOduration=33.894232559 podStartE2EDuration="33.894232559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:03.893449494 +0000 UTC m=+306.574382250" watchObservedRunningTime="2026-09-10 00:05:03.894232559 +0000 UTC m=+306.575165514" Sep 10 00:05:04.812168 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:04.812125 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:04.814555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:04.814531 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:04.879508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:04.879476 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:04.880920 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:04.880892 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64cc444bc6-mqvdc" Sep 10 00:05:05.822173 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:05.822138 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:05:12.390511 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.390469 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:05:12.394734 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.394701 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.396870 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.396847 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 00:05:12.398055 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.397927 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 00:05:12.398055 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.397942 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 00:05:12.398055 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.397954 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 00:05:12.398274 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.398054 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 00:05:12.398274 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.398076 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2tdg4\"" Sep 10 00:05:12.401981 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.401960 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 00:05:12.403599 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.403578 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:05:12.527109 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527069 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527176 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527214 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np8g7\" (UniqueName: \"kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527497 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.527497 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.527370 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628332 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np8g7\" (UniqueName: \"kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628542 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628542 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628542 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628565 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.628711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.628644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.629311 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.629285 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.629440 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.629392 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.629440 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.629402 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.629582 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.629544 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.631380 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.631359 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.631531 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.631472 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.635665 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.635641 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np8g7\" (UniqueName: \"kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7\") pod \"console-77976b99d6-f6vx4\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:12.706195 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:12.706113 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:13.619588 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.619482 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:05:13.622651 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:13.622620 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c0c072_1c54_46d5_a0c2_7d41d65e8215.slice/crio-f66e19d40bc9ec6908c3b6b66f34978a4ba08e3858b2d43022b15bde8cb06b85 WatchSource:0}: Error finding container f66e19d40bc9ec6908c3b6b66f34978a4ba08e3858b2d43022b15bde8cb06b85: Status 404 returned error can't find the container with id f66e19d40bc9ec6908c3b6b66f34978a4ba08e3858b2d43022b15bde8cb06b85 Sep 10 00:05:13.913815 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.913729 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" event={"ID":"8157b04a-9036-4a96-a570-eb8d0e22dba5","Type":"ContainerStarted","Data":"95d0340fe4b80b9cc0dd94e08a5ca6f87bd6f62f1c2592d996936bb26ea05946"} Sep 10 00:05:13.915679 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.915646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-m6248" event={"ID":"2c1ae540-1b98-45a5-93ae-d7eab119984c","Type":"ContainerStarted","Data":"a19c6bd50f61537e9e758cd7752d063cb440ccfa8a5045b6d300e058c403d556"} Sep 10 00:05:13.916096 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.916077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:05:13.917841 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.917494 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77976b99d6-f6vx4" event={"ID":"72c0c072-1c54-46d5-a0c2-7d41d65e8215","Type":"ContainerStarted","Data":"f66e19d40bc9ec6908c3b6b66f34978a4ba08e3858b2d43022b15bde8cb06b85"} Sep 10 00:05:13.929821 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.929770 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xtpll" podStartSLOduration=33.584188988 podStartE2EDuration="43.929751373s" totalImagesPullingTime="10.345562385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:13.929681838 +0000 UTC m=+316.610614604" watchObservedRunningTime="2026-09-10 00:05:13.929751373 +0000 UTC m=+316.610684116" Sep 10 00:05:13.931944 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.931920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-m6248" Sep 10 00:05:13.946622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:13.946567 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-m6248" podStartSLOduration=1.408096168 podStartE2EDuration="17.946552277s" totalImagesPullingTime="16.538456109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:13.945053818 +0000 UTC m=+316.625986565" watchObservedRunningTime="2026-09-10 00:05:13.946552277 +0000 UTC m=+316.627485022" Sep 10 00:05:16.921102 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.919822 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8qp5q"] Sep 10 00:05:16.960534 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.960478 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8qp5q"] Sep 10 00:05:16.960745 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.960645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:16.963008 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.962985 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 00:05:16.964446 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.963924 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 00:05:16.964446 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.963941 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 00:05:16.964446 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:16.964320 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wsncx\"" Sep 10 00:05:17.069705 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.069648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.069881 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.069758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52d2f86a-89d5-4bd1-932a-1037ed420e57-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.069881 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.069802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.069881 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.069847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mdrn\" (UniqueName: \"kubernetes.io/projected/52d2f86a-89d5-4bd1-932a-1037ed420e57-kube-api-access-6mdrn\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.170944 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.170899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.171137 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.170987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52d2f86a-89d5-4bd1-932a-1037ed420e57-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.171137 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.171041 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.171137 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:05:17.171074 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 10 00:05:17.171304 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:05:17.171146 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls podName:52d2f86a-89d5-4bd1-932a-1037ed420e57 nodeName:}" failed. No retries permitted until 2026-09-10 00:05:17.671125457 +0000 UTC m=+320.352058193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-8qp5q" (UID: "52d2f86a-89d5-4bd1-932a-1037ed420e57") : secret "prometheus-operator-tls" not found Sep 10 00:05:17.171304 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.171249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mdrn\" (UniqueName: \"kubernetes.io/projected/52d2f86a-89d5-4bd1-932a-1037ed420e57-kube-api-access-6mdrn\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.171745 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.171725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52d2f86a-89d5-4bd1-932a-1037ed420e57-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.173707 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.173687 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.181166 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.181137 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mdrn\" (UniqueName: \"kubernetes.io/projected/52d2f86a-89d5-4bd1-932a-1037ed420e57-kube-api-access-6mdrn\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.675939 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.675903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.678655 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.678629 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d2f86a-89d5-4bd1-932a-1037ed420e57-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8qp5q\" (UID: \"52d2f86a-89d5-4bd1-932a-1037ed420e57\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.736902 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.736874 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fff784fd-nttpn"] Sep 10 00:05:17.737177 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:05:17.737153 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" podUID="d8f88911-2bce-4790-a5f7-503d7a4956e3" Sep 10 00:05:17.873897 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.873821 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" Sep 10 00:05:17.942063 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.941294 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:05:17.942063 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.941368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77976b99d6-f6vx4" event={"ID":"72c0c072-1c54-46d5-a0c2-7d41d65e8215","Type":"ContainerStarted","Data":"f407ad7aabb6e5489ff4910bcade55f44a4720bbc0d0c3897b3691bc4d1dbd1f"} Sep 10 00:05:17.971864 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.971838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:05:17.976932 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:17.976880 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77976b99d6-f6vx4" podStartSLOduration=2.09338368 podStartE2EDuration="5.9768651s" totalImagesPullingTime="3.88348142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:17.975474677 +0000 UTC m=+320.656407446" watchObservedRunningTime="2026-09-10 00:05:17.9768651 +0000 UTC m=+320.657797841" Sep 10 00:05:18.078634 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078596 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.078839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078655 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.078839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078697 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.078839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078749 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.078839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078783 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.078839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078823 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnn6x\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.079116 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078850 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted\") pod \"d8f88911-2bce-4790-a5f7-503d7a4956e3\" (UID: \"d8f88911-2bce-4790-a5f7-503d7a4956e3\") " Sep 10 00:05:18.079116 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.078965 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:05:18.079512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.079325 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-certificates\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.079512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.079417 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:05:18.080273 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.080248 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:05:18.081640 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.081609 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:05:18.081956 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.081931 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:05:18.082226 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.082199 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:05:18.082326 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.082281 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x" (OuterVolumeSpecName: "kube-api-access-vnn6x") pod "d8f88911-2bce-4790-a5f7-503d7a4956e3" (UID: "d8f88911-2bce-4790-a5f7-503d7a4956e3"). InnerVolumeSpecName "kube-api-access-vnn6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:05:18.161944 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.161858 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8qp5q"] Sep 10 00:05:18.165044 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:18.165016 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d2f86a_89d5_4bd1_932a_1037ed420e57.slice/crio-2cbd2dd98b59d58aa2703ff591f2472df53096b91564cd25fc5021435c88175c WatchSource:0}: Error finding container 2cbd2dd98b59d58aa2703ff591f2472df53096b91564cd25fc5021435c88175c: Status 404 returned error can't find the container with id 2cbd2dd98b59d58aa2703ff591f2472df53096b91564cd25fc5021435c88175c Sep 10 00:05:18.180317 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180291 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-image-registry-private-configuration\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.180424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180319 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-bound-sa-token\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.180424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180334 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d8f88911-2bce-4790-a5f7-503d7a4956e3-installation-pull-secrets\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.180424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180348 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8f88911-2bce-4790-a5f7-503d7a4956e3-trusted-ca\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.180424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180362 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnn6x\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-kube-api-access-vnn6x\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.180424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.180375 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d8f88911-2bce-4790-a5f7-503d7a4956e3-ca-trust-extracted\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:18.947489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.947455 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fff784fd-nttpn" Sep 10 00:05:18.948079 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.947617 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" event={"ID":"52d2f86a-89d5-4bd1-932a-1037ed420e57","Type":"ContainerStarted","Data":"2cbd2dd98b59d58aa2703ff591f2472df53096b91564cd25fc5021435c88175c"} Sep 10 00:05:18.980403 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.980302 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fff784fd-nttpn"] Sep 10 00:05:18.983931 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:18.983880 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76fff784fd-nttpn"] Sep 10 00:05:19.089507 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:19.089473 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d8f88911-2bce-4790-a5f7-503d7a4956e3-registry-tls\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:05:19.827909 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:19.827867 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f88911-2bce-4790-a5f7-503d7a4956e3" path="/var/lib/kubelet/pods/d8f88911-2bce-4790-a5f7-503d7a4956e3/volumes" Sep 10 00:05:20.957485 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:20.957447 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" event={"ID":"52d2f86a-89d5-4bd1-932a-1037ed420e57","Type":"ContainerStarted","Data":"9c9ccc933cbe3c38af61ae4df11b1f803b83c92e2ea95a837ed88bfd7ba60359"} Sep 10 00:05:20.957930 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:20.957531 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" event={"ID":"52d2f86a-89d5-4bd1-932a-1037ed420e57","Type":"ContainerStarted","Data":"bae79712b1e57b969254a1f7b0ce8f6399d5e6ba0f862e214aab01ee2d635667"} Sep 10 00:05:20.973664 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:20.973609 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-8qp5q" podStartSLOduration=3.002393947 podStartE2EDuration="4.973588179s" totalImagesPullingTime="1.971194232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:20.973111477 +0000 UTC m=+323.654044244" watchObservedRunningTime="2026-09-10 00:05:20.973588179 +0000 UTC m=+323.654520918" Sep 10 00:05:22.707045 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:22.707002 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:22.707419 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:22.707059 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:22.711910 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:22.711892 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:22.969160 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:22.969069 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:05:23.254628 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.254595 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc"] Sep 10 00:05:23.285740 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.285701 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc"] Sep 10 00:05:23.285912 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.285744 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.285912 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.285781 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-svbxb"] Sep 10 00:05:23.289250 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.289220 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 00:05:23.289250 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.289218 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 00:05:23.289485 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.289472 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f8tzn\"" Sep 10 00:05:23.308061 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.308035 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.310669 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.310650 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fpmfn\"" Sep 10 00:05:23.310876 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.310860 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 00:05:23.310930 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.310878 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 00:05:23.310969 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.310927 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 00:05:23.427162 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427131 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-accelerators-collector-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-metrics-client-ca\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.427350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-root\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427350 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7hqs\" (UniqueName: \"kubernetes.io/projected/66002a5d-bdc8-4612-afbf-f58a1a7e010a-kube-api-access-w7hqs\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427388 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7531d018-aa86-4eb8-808a-093dedfac6c7-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-sys\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-tls\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427483 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-textfile\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427504 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427723 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-wtmp\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.427723 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.427616 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7d2n\" (UniqueName: \"kubernetes.io/projected/7531d018-aa86-4eb8-808a-093dedfac6c7-kube-api-access-c7d2n\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.528847 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-sys\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.528847 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.528847 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528828 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-tls\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.528847 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-textfile\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528890 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-sys\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528923 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-wtmp\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7d2n\" (UniqueName: \"kubernetes.io/projected/7531d018-aa86-4eb8-808a-093dedfac6c7-kube-api-access-c7d2n\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.528986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-accelerators-collector-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:05:23.529011 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529030 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-metrics-client-ca\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:05:23.529079 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls podName:7531d018-aa86-4eb8-808a-093dedfac6c7 nodeName:}" failed. No retries permitted until 2026-09-10 00:05:24.029060534 +0000 UTC m=+326.709993272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-mq5qc" (UID: "7531d018-aa86-4eb8-808a-093dedfac6c7") : secret "openshift-state-metrics-tls" not found Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529098 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-wtmp\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-root\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529145 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7hqs\" (UniqueName: \"kubernetes.io/projected/66002a5d-bdc8-4612-afbf-f58a1a7e010a-kube-api-access-w7hqs\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529772 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529179 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-textfile\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529772 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529196 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7531d018-aa86-4eb8-808a-093dedfac6c7-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.529772 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66002a5d-bdc8-4612-afbf-f58a1a7e010a-root\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529944 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529777 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-metrics-client-ca\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.529997 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.529982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7531d018-aa86-4eb8-808a-093dedfac6c7-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.530052 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.530020 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-accelerators-collector-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.531391 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.531369 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.531511 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.531490 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.531572 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.531495 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66002a5d-bdc8-4612-afbf-f58a1a7e010a-node-exporter-tls\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.536958 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.536930 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7d2n\" (UniqueName: \"kubernetes.io/projected/7531d018-aa86-4eb8-808a-093dedfac6c7-kube-api-access-c7d2n\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:23.536958 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.536944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7hqs\" (UniqueName: \"kubernetes.io/projected/66002a5d-bdc8-4612-afbf-f58a1a7e010a-kube-api-access-w7hqs\") pod \"node-exporter-svbxb\" (UID: \"66002a5d-bdc8-4612-afbf-f58a1a7e010a\") " pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.616908 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.616871 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-svbxb" Sep 10 00:05:23.659824 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:23.659790 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66002a5d_bdc8_4612_afbf_f58a1a7e010a.slice/crio-44f5ae7f86d4ff6584db39ca668767268890f8174954b22f2292d77e0f48d093 WatchSource:0}: Error finding container 44f5ae7f86d4ff6584db39ca668767268890f8174954b22f2292d77e0f48d093: Status 404 returned error can't find the container with id 44f5ae7f86d4ff6584db39ca668767268890f8174954b22f2292d77e0f48d093 Sep 10 00:05:23.969368 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:23.969297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svbxb" event={"ID":"66002a5d-bdc8-4612-afbf-f58a1a7e010a","Type":"ContainerStarted","Data":"44f5ae7f86d4ff6584db39ca668767268890f8174954b22f2292d77e0f48d093"} Sep 10 00:05:24.033729 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.033691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:24.036178 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.036150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7531d018-aa86-4eb8-808a-093dedfac6c7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-mq5qc\" (UID: \"7531d018-aa86-4eb8-808a-093dedfac6c7\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:24.195051 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.195006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" Sep 10 00:05:24.489417 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.489389 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc"] Sep 10 00:05:24.491936 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:24.491907 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7531d018_aa86_4eb8_808a_093dedfac6c7.slice/crio-0728023565ec8481ca03cd240591b1d3e30a38440db23be9a626fcaab1b54710 WatchSource:0}: Error finding container 0728023565ec8481ca03cd240591b1d3e30a38440db23be9a626fcaab1b54710: Status 404 returned error can't find the container with id 0728023565ec8481ca03cd240591b1d3e30a38440db23be9a626fcaab1b54710 Sep 10 00:05:24.974579 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.974551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" event={"ID":"7531d018-aa86-4eb8-808a-093dedfac6c7","Type":"ContainerStarted","Data":"c0227f13894ca5a4f516ec066f30836edd0043bb2d331725ea93eecc8b11aad9"} Sep 10 00:05:24.974914 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:24.974607 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" event={"ID":"7531d018-aa86-4eb8-808a-093dedfac6c7","Type":"ContainerStarted","Data":"0728023565ec8481ca03cd240591b1d3e30a38440db23be9a626fcaab1b54710"} Sep 10 00:05:25.979988 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:25.979956 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" event={"ID":"7531d018-aa86-4eb8-808a-093dedfac6c7","Type":"ContainerStarted","Data":"c9a705b44bc63e36ab387091aefa6f41e9e9c64c07d77597efe514b6b4d38f30"} Sep 10 00:05:25.981557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:25.981508 2582 generic.go:356] "Generic (PLEG): container finished" podID="66002a5d-bdc8-4612-afbf-f58a1a7e010a" containerID="4ccb9284343cf1627d1b84d477c95f3ebdfb5b07e33e3cbf077bf9e416600bc1" exitCode=0 Sep 10 00:05:25.981670 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:25.981566 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svbxb" event={"ID":"66002a5d-bdc8-4612-afbf-f58a1a7e010a","Type":"ContainerDied","Data":"4ccb9284343cf1627d1b84d477c95f3ebdfb5b07e33e3cbf077bf9e416600bc1"} Sep 10 00:05:26.247705 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.247635 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64f469f46-zs2lw"] Sep 10 00:05:26.263827 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.263800 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.267815 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.267790 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 00:05:26.267959 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.267790 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 00:05:26.267959 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.267842 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 00:05:26.268118 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.268104 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q5bj7\"" Sep 10 00:05:26.268310 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.268297 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 00:05:26.268659 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.268641 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-962lg3ddjs2vk\"" Sep 10 00:05:26.269189 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.269002 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 00:05:26.274732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.272873 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64f469f46-zs2lw"] Sep 10 00:05:26.355056 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355243 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355141 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc5hd\" (UniqueName: \"kubernetes.io/projected/af94e8ae-f947-424f-ada7-939e161f203d-kube-api-access-mc5hd\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355243 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355243 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355415 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355415 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355318 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af94e8ae-f947-424f-ada7-939e161f203d-metrics-client-ca\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355415 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-grpc-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.355415 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.355392 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456051 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af94e8ae-f947-424f-ada7-939e161f203d-metrics-client-ca\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456051 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-grpc-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456240 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456240 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456116 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc5hd\" (UniqueName: \"kubernetes.io/projected/af94e8ae-f947-424f-ada7-939e161f203d-kube-api-access-mc5hd\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456283 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456409 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456409 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.456894 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.456868 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af94e8ae-f947-424f-ada7-939e161f203d-metrics-client-ca\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.458866 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.458839 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.458967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.458898 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.459158 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.459136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.459245 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.459231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.459292 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.459241 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.459292 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.459276 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af94e8ae-f947-424f-ada7-939e161f203d-secret-grpc-tls\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.463611 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.463589 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc5hd\" (UniqueName: \"kubernetes.io/projected/af94e8ae-f947-424f-ada7-939e161f203d-kube-api-access-mc5hd\") pod \"thanos-querier-64f469f46-zs2lw\" (UID: \"af94e8ae-f947-424f-ada7-939e161f203d\") " pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.579503 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.579420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:26.878162 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.878084 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64f469f46-zs2lw"] Sep 10 00:05:26.881030 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:26.881006 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf94e8ae_f947_424f_ada7_939e161f203d.slice/crio-792796dfb531b3db5685d8c454bb9b4b471033f7884cef27c694f6594e5d75c1 WatchSource:0}: Error finding container 792796dfb531b3db5685d8c454bb9b4b471033f7884cef27c694f6594e5d75c1: Status 404 returned error can't find the container with id 792796dfb531b3db5685d8c454bb9b4b471033f7884cef27c694f6594e5d75c1 Sep 10 00:05:26.987135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.987104 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svbxb" event={"ID":"66002a5d-bdc8-4612-afbf-f58a1a7e010a","Type":"ContainerStarted","Data":"e67615a843e8e914014d8ab237e4c0485c5feb43c9a03e8e8faf914a2790b98f"} Sep 10 00:05:26.987585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.987158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svbxb" event={"ID":"66002a5d-bdc8-4612-afbf-f58a1a7e010a","Type":"ContainerStarted","Data":"b69841c35e0e4b626536cc0044dab33b9544a03399982e1a5691bf23a71edf09"} Sep 10 00:05:26.988244 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:26.988221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"792796dfb531b3db5685d8c454bb9b4b471033f7884cef27c694f6594e5d75c1"} Sep 10 00:05:27.006922 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.006876 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-svbxb" podStartSLOduration=2.387245032 podStartE2EDuration="4.006861889s" totalImagesPullingTime="1.542606892s" totalInitContainerRuntime="77.009965ms" isStatefulPod=true podCreationTimestamp="2026-09-10 00:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:27.005448436 +0000 UTC m=+329.686381197" watchObservedRunningTime="2026-09-10 00:05:27.006861889 +0000 UTC m=+329.687794631" Sep 10 00:05:27.662970 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.662938 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-748bbb5d6c-j9qj2"] Sep 10 00:05:27.678472 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.678434 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-748bbb5d6c-j9qj2"] Sep 10 00:05:27.678653 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.678562 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.681142 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.681114 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 00:05:27.681257 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.681124 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 00:05:27.681873 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.681854 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4eakit2n3913k\"" Sep 10 00:05:27.682130 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.682094 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d9284\"" Sep 10 00:05:27.682130 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.682105 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 00:05:27.682288 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.682139 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 00:05:27.773271 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-metrics-server-audit-profiles\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773301 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-client-certs\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-tls\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-audit-log\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773437 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-client-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.773684 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.773503 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4kj2\" (UniqueName: \"kubernetes.io/projected/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-kube-api-access-x4kj2\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874115 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874084 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4kj2\" (UniqueName: \"kubernetes.io/projected/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-kube-api-access-x4kj2\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874144 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-metrics-server-audit-profiles\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874210 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-client-certs\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-tls\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874488 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-audit-log\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874580 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-client-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874771 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874747 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-audit-log\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.874986 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.874962 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.875260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.875230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-metrics-server-audit-profiles\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.876833 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.876808 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-tls\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.876926 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.876909 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-secret-metrics-server-client-certs\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.876967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.876927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-client-ca-bundle\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.882145 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.882129 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4kj2\" (UniqueName: \"kubernetes.io/projected/f0c5ddeb-f017-45fa-8b69-5e9e130420e9-kube-api-access-x4kj2\") pod \"metrics-server-748bbb5d6c-j9qj2\" (UID: \"f0c5ddeb-f017-45fa-8b69-5e9e130420e9\") " pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:27.989542 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:27.989326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:28.287137 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:28.286992 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-748bbb5d6c-j9qj2"] Sep 10 00:05:28.290296 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:05:28.290264 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c5ddeb_f017_45fa_8b69_5e9e130420e9.slice/crio-913b0d28510d33c2992a9218bcaf91d5415ac085efa5f1c426297df223810622 WatchSource:0}: Error finding container 913b0d28510d33c2992a9218bcaf91d5415ac085efa5f1c426297df223810622: Status 404 returned error can't find the container with id 913b0d28510d33c2992a9218bcaf91d5415ac085efa5f1c426297df223810622 Sep 10 00:05:29.001686 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:29.001600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" event={"ID":"7531d018-aa86-4eb8-808a-093dedfac6c7","Type":"ContainerStarted","Data":"8c74a4a138890219d669a86a147d1287ddd8ffe12d25abe6363847fb974d6bc2"} Sep 10 00:05:29.003054 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:29.003015 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" event={"ID":"f0c5ddeb-f017-45fa-8b69-5e9e130420e9","Type":"ContainerStarted","Data":"913b0d28510d33c2992a9218bcaf91d5415ac085efa5f1c426297df223810622"} Sep 10 00:05:29.018926 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:29.018867 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-mq5qc" podStartSLOduration=3.066986829 podStartE2EDuration="6.018847299s" totalImagesPullingTime="2.95186047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:29.017004574 +0000 UTC m=+331.697937317" watchObservedRunningTime="2026-09-10 00:05:29.018847299 +0000 UTC m=+331.699780042" Sep 10 00:05:31.014666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:31.014630 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"3011880412f25b7ab8eac066e78224842032ec458f66fbf2b534bcfc00e5bcca"} Sep 10 00:05:31.015071 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:31.014698 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"d1dbaa7ec99e83a79f84430f5d4be2dde28506794d9c19e55a0c117a4bcd0d19"} Sep 10 00:05:31.015071 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:31.014743 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"478d0583596e02deca96e80a572c043e43058e36317f6038937368e1443c5761"} Sep 10 00:05:31.016035 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:31.015998 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" event={"ID":"f0c5ddeb-f017-45fa-8b69-5e9e130420e9","Type":"ContainerStarted","Data":"2e5bf156ae52376c6e305ac74d36f91ada0ed1f17d5b39b98851e81becf9018c"} Sep 10 00:05:31.031705 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:31.031629 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" podStartSLOduration=1.7965429149999999 podStartE2EDuration="4.03161327s" totalImagesPullingTime="2.235070355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:31.030110852 +0000 UTC m=+333.711043613" watchObservedRunningTime="2026-09-10 00:05:31.03161327 +0000 UTC m=+333.712546011" Sep 10 00:05:33.026478 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:33.026443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"966fedef2b108b699a899537add55555daf846650f4ee0dc4eff66a8598ed4b1"} Sep 10 00:05:33.026907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:33.026558 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"a0fdbfe70f075bc81ab16bd1fb05a9d0e59f571259104320767146c234eb1684"} Sep 10 00:05:33.026907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:33.026594 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:33.026907 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:33.026623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" event={"ID":"af94e8ae-f947-424f-ada7-939e161f203d","Type":"ContainerStarted","Data":"995c2a3510777620fb3af1c414725058de11bddf2d61ddf1924605d000a9ceb3"} Sep 10 00:05:33.046017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:33.045965 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" podStartSLOduration=2.110057552 podStartE2EDuration="7.045949374s" totalImagesPullingTime="4.935891822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:05:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:05:33.044734933 +0000 UTC m=+335.725667678" watchObservedRunningTime="2026-09-10 00:05:33.045949374 +0000 UTC m=+335.726882116" Sep 10 00:05:39.035833 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:39.035802 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64f469f46-zs2lw" Sep 10 00:05:43.942452 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:43.942418 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:05:47.990431 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:47.990394 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:05:47.990843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:05:47.990456 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:06:07.995598 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:07.995526 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:06:07.999283 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:07.999262 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-748bbb5d6c-j9qj2" Sep 10 00:06:08.966097 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:08.966037 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77976b99d6-f6vx4" podUID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" containerName="console" containerID="cri-o://f407ad7aabb6e5489ff4910bcade55f44a4720bbc0d0c3897b3691bc4d1dbd1f" gracePeriod=15 Sep 10 00:06:09.165752 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.165722 2582 generic.go:356] "Generic (PLEG): container finished" podID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" containerID="f407ad7aabb6e5489ff4910bcade55f44a4720bbc0d0c3897b3691bc4d1dbd1f" exitCode=0 Sep 10 00:06:09.166179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.165782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77976b99d6-f6vx4" event={"ID":"72c0c072-1c54-46d5-a0c2-7d41d65e8215","Type":"ContainerDied","Data":"f407ad7aabb6e5489ff4910bcade55f44a4720bbc0d0c3897b3691bc4d1dbd1f"} Sep 10 00:06:09.235410 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.235389 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:06:09.351151 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351115 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351201 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351221 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351242 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np8g7\" (UniqueName: \"kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351270 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351290 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351321 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle\") pod \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\" (UID: \"72c0c072-1c54-46d5-a0c2-7d41d65e8215\") " Sep 10 00:06:09.351695 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351667 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca" (OuterVolumeSpecName: "service-ca") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:06:09.351807 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351785 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config" (OuterVolumeSpecName: "console-config") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:06:09.351882 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351804 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:06:09.351930 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.351906 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:06:09.353492 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.353467 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:06:09.353937 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.353915 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:06:09.354017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.353948 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7" (OuterVolumeSpecName: "kube-api-access-np8g7") pod "72c0c072-1c54-46d5-a0c2-7d41d65e8215" (UID: "72c0c072-1c54-46d5-a0c2-7d41d65e8215"). InnerVolumeSpecName "kube-api-access-np8g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:06:09.452314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452277 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-oauth-config\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452308 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-config\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452318 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-trusted-ca-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452326 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-service-ca\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452633 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452335 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72c0c072-1c54-46d5-a0c2-7d41d65e8215-oauth-serving-cert\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452633 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452343 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72c0c072-1c54-46d5-a0c2-7d41d65e8215-console-serving-cert\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:09.452633 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:09.452351 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np8g7\" (UniqueName: \"kubernetes.io/projected/72c0c072-1c54-46d5-a0c2-7d41d65e8215-kube-api-access-np8g7\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:06:10.170660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.170574 2582 generic.go:356] "Generic (PLEG): container finished" podID="60945d27-560a-4451-8173-ea86bcb8812c" containerID="0b94977f18e0b158fc017e571087a8ca5ea789fe5791fd77c75e98f2dcc3a86e" exitCode=0 Sep 10 00:06:10.170660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.170652 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" event={"ID":"60945d27-560a-4451-8173-ea86bcb8812c","Type":"ContainerDied","Data":"0b94977f18e0b158fc017e571087a8ca5ea789fe5791fd77c75e98f2dcc3a86e"} Sep 10 00:06:10.171233 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.171008 2582 scope.go:117] "RemoveContainer" containerID="0b94977f18e0b158fc017e571087a8ca5ea789fe5791fd77c75e98f2dcc3a86e" Sep 10 00:06:10.172058 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.172034 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77976b99d6-f6vx4" Sep 10 00:06:10.172155 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.172031 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77976b99d6-f6vx4" event={"ID":"72c0c072-1c54-46d5-a0c2-7d41d65e8215","Type":"ContainerDied","Data":"f66e19d40bc9ec6908c3b6b66f34978a4ba08e3858b2d43022b15bde8cb06b85"} Sep 10 00:06:10.172249 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.172170 2582 scope.go:117] "RemoveContainer" containerID="f407ad7aabb6e5489ff4910bcade55f44a4720bbc0d0c3897b3691bc4d1dbd1f" Sep 10 00:06:10.197378 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.197353 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:06:10.202553 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.202529 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77976b99d6-f6vx4"] Sep 10 00:06:10.863543 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.863485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:06:10.865817 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.865785 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb4843b-acb3-45bd-98c1-11acccb93f3a-metrics-certs\") pod \"network-metrics-daemon-bxlwt\" (UID: \"2cb4843b-acb3-45bd-98c1-11acccb93f3a\") " pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:06:10.925404 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.925373 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s66cg\"" Sep 10 00:06:10.934098 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:10.934076 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bxlwt" Sep 10 00:06:11.177225 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:11.177136 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2zzfq" event={"ID":"60945d27-560a-4451-8173-ea86bcb8812c","Type":"ContainerStarted","Data":"e5186ca953022002ef83155f11fdcd2bb6b95dd5a53aec8a972598bd015ee95c"} Sep 10 00:06:11.204793 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:11.204762 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bxlwt"] Sep 10 00:06:11.207937 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:06:11.207907 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb4843b_acb3_45bd_98c1_11acccb93f3a.slice/crio-8226eb7993fd1d478395026defdc4d56507c73a378bf46ddb6c8bf99e990573a WatchSource:0}: Error finding container 8226eb7993fd1d478395026defdc4d56507c73a378bf46ddb6c8bf99e990573a: Status 404 returned error can't find the container with id 8226eb7993fd1d478395026defdc4d56507c73a378bf46ddb6c8bf99e990573a Sep 10 00:06:11.827311 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:11.827278 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" path="/var/lib/kubelet/pods/72c0c072-1c54-46d5-a0c2-7d41d65e8215/volumes" Sep 10 00:06:12.183172 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:12.183089 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bxlwt" event={"ID":"2cb4843b-acb3-45bd-98c1-11acccb93f3a","Type":"ContainerStarted","Data":"8226eb7993fd1d478395026defdc4d56507c73a378bf46ddb6c8bf99e990573a"} Sep 10 00:06:13.188812 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:13.188776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bxlwt" event={"ID":"2cb4843b-acb3-45bd-98c1-11acccb93f3a","Type":"ContainerStarted","Data":"e4ea79311be42776112ef54e62fbc0f0511b43bd670bf9006eddc88de9d31db9"} Sep 10 00:06:13.189199 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:13.188832 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bxlwt" event={"ID":"2cb4843b-acb3-45bd-98c1-11acccb93f3a","Type":"ContainerStarted","Data":"05b19c5f5caead9cd9136efba7c54c87424dae934034256621caf6d59095eb15"} Sep 10 00:06:13.203502 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:13.203452 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bxlwt" podStartSLOduration=373.032135396 podStartE2EDuration="6m14.20343593s" totalImagesPullingTime="1.171300534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:06:13.202385725 +0000 UTC m=+375.883318492" watchObservedRunningTime="2026-09-10 00:06:13.20343593 +0000 UTC m=+375.884368671" Sep 10 00:06:41.780848 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:06:41.780801 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" podUID="b03b6f71-f814-4d17-bcd7-f73de5046adf" Sep 10 00:06:41.781303 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:06:41.781017 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-c7r5j" podUID="d5e4505d-29bf-4961-9e60-0e37f6b40d54" Sep 10 00:06:42.301044 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:42.301013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:06:42.301044 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:42.301053 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:06:43.923771 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:43.923736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:06:43.926197 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:43.926164 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b03b6f71-f814-4d17-bcd7-f73de5046adf-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fgc5x\" (UID: \"b03b6f71-f814-4d17-bcd7-f73de5046adf\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:06:44.024310 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.024273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:06:44.026629 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.026608 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e4505d-29bf-4961-9e60-0e37f6b40d54-metrics-tls\") pod \"dns-default-c7r5j\" (UID: \"d5e4505d-29bf-4961-9e60-0e37f6b40d54\") " pod="openshift-dns/dns-default-c7r5j" Sep 10 00:06:44.104369 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.104341 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f86ng\"" Sep 10 00:06:44.104369 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.104341 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gg685\"" Sep 10 00:06:44.112293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.112271 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" Sep 10 00:06:44.112399 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.112299 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:06:44.394691 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.394572 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fgc5x"] Sep 10 00:06:44.401502 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:44.401141 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c7r5j"] Sep 10 00:06:44.404403 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:06:44.404380 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e4505d_29bf_4961_9e60_0e37f6b40d54.slice/crio-9469a449322ceb0258678950a8bd7b6514abc181445d2944e0eb8e7d3b02c013 WatchSource:0}: Error finding container 9469a449322ceb0258678950a8bd7b6514abc181445d2944e0eb8e7d3b02c013: Status 404 returned error can't find the container with id 9469a449322ceb0258678950a8bd7b6514abc181445d2944e0eb8e7d3b02c013 Sep 10 00:06:45.313416 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:45.313373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7r5j" event={"ID":"d5e4505d-29bf-4961-9e60-0e37f6b40d54","Type":"ContainerStarted","Data":"9469a449322ceb0258678950a8bd7b6514abc181445d2944e0eb8e7d3b02c013"} Sep 10 00:06:45.314575 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:45.314548 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" event={"ID":"b03b6f71-f814-4d17-bcd7-f73de5046adf","Type":"ContainerStarted","Data":"40135f92a276e291c0baa1a944b781276125380d53ab67317fc2478014c8bb6c"} Sep 10 00:06:47.324769 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.324731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7r5j" event={"ID":"d5e4505d-29bf-4961-9e60-0e37f6b40d54","Type":"ContainerStarted","Data":"e7675a28139285f14fdb6275b016338690460fe4d8b3719ced93aa71c27dda4e"} Sep 10 00:06:47.325260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.324787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7r5j" event={"ID":"d5e4505d-29bf-4961-9e60-0e37f6b40d54","Type":"ContainerStarted","Data":"16eb4014852a4a9b23e8cbdee3b3213f6de0f7e82801b3ed1b0c409811346930"} Sep 10 00:06:47.325260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.324869 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:06:47.326285 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.326255 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" event={"ID":"b03b6f71-f814-4d17-bcd7-f73de5046adf","Type":"ContainerStarted","Data":"cc306a9000aa35169fc5b63310b91b7cd7881ba316eba6f646a93f5f440613a1"} Sep 10 00:06:47.341250 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.341203 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c7r5j" podStartSLOduration=374.014380063 podStartE2EDuration="6m16.341189326s" totalImagesPullingTime="2.326809263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:06:47.340374837 +0000 UTC m=+410.021307590" watchObservedRunningTime="2026-09-10 00:06:47.341189326 +0000 UTC m=+410.022122075" Sep 10 00:06:47.356967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:47.356929 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-fgc5x" podStartSLOduration=375.056562098 podStartE2EDuration="6m17.356914362s" totalImagesPullingTime="2.300352264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:06:47.355994785 +0000 UTC m=+410.036927555" watchObservedRunningTime="2026-09-10 00:06:47.356914362 +0000 UTC m=+410.037847104" Sep 10 00:06:52.563640 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.563604 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:06:52.564135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.564080 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" containerName="console" Sep 10 00:06:52.564135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.564100 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" containerName="console" Sep 10 00:06:52.564279 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.564196 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="72c0c072-1c54-46d5-a0c2-7d41d65e8215" containerName="console" Sep 10 00:06:52.566264 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.566184 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.569273 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569243 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 00:06:52.569381 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569281 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 00:06:52.569381 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569301 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2tdg4\"" Sep 10 00:06:52.569498 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569243 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 00:06:52.569660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569645 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 00:06:52.569719 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.569708 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 00:06:52.575095 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.573560 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 00:06:52.575897 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.575874 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:06:52.696791 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696819 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696888 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvt7g\" (UniqueName: \"kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696905 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.696947 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.696921 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797490 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797610 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvt7g\" (UniqueName: \"kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.797889 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.797795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.798322 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.798297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.798430 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.798298 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.798600 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.798576 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.798664 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.798650 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.800501 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.800479 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.800650 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.800632 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.804648 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.804628 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvt7g\" (UniqueName: \"kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g\") pod \"console-84dc48bd5-tn2b2\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:52.878380 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:52.878298 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:06:53.146737 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:53.146653 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:06:53.350383 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:53.350343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dc48bd5-tn2b2" event={"ID":"8bc623d6-5ec0-4483-8617-25edf1fce050","Type":"ContainerStarted","Data":"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f"} Sep 10 00:06:53.350541 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:53.350397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dc48bd5-tn2b2" event={"ID":"8bc623d6-5ec0-4483-8617-25edf1fce050","Type":"ContainerStarted","Data":"444690284d49e56cf206006eeaef89e5cf71d5bdf09a3cd8f42a5a339943eb4a"} Sep 10 00:06:53.367083 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:53.367033 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84dc48bd5-tn2b2" podStartSLOduration=1.367014144 podStartE2EDuration="1.367014144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:06:53.36565572 +0000 UTC m=+416.046588495" watchObservedRunningTime="2026-09-10 00:06:53.367014144 +0000 UTC m=+416.047946888" Sep 10 00:06:57.332786 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:06:57.332749 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c7r5j" Sep 10 00:07:02.879124 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:07:02.879081 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:07:02.879508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:07:02.879164 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:07:02.883694 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:07:02.883669 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:07:03.391367 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:07:03.391341 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:08:03.619045 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:03.619011 2582 generic.go:356] "Generic (PLEG): container finished" podID="41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" containerID="85bd5e86bc5caab40fea72dcc7e6beb4206a35c0d60dcb46d8e35b02530db187" exitCode=0 Sep 10 00:08:03.619444 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:03.619082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29816640-2r965" event={"ID":"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e","Type":"ContainerDied","Data":"85bd5e86bc5caab40fea72dcc7e6beb4206a35c0d60dcb46d8e35b02530db187"} Sep 10 00:08:04.744925 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.744900 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:08:04.890543 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.890431 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4d8l\" (UniqueName: \"kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l\") pod \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " Sep 10 00:08:04.890711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.890558 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca\") pod \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\" (UID: \"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e\") " Sep 10 00:08:04.890902 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.890880 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca" (OuterVolumeSpecName: "serviceca") pod "41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" (UID: "41bbd9ae-02c6-43bb-bf1e-8afaaceb980e"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:08:04.892681 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.892661 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l" (OuterVolumeSpecName: "kube-api-access-w4d8l") pod "41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" (UID: "41bbd9ae-02c6-43bb-bf1e-8afaaceb980e"). InnerVolumeSpecName "kube-api-access-w4d8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:08:04.992032 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.991987 2582 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-serviceca\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:08:04.992032 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:04.992027 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4d8l\" (UniqueName: \"kubernetes.io/projected/41bbd9ae-02c6-43bb-bf1e-8afaaceb980e-kube-api-access-w4d8l\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:08:05.627913 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:05.627885 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29816640-2r965" Sep 10 00:08:05.628080 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:05.627884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29816640-2r965" event={"ID":"41bbd9ae-02c6-43bb-bf1e-8afaaceb980e","Type":"ContainerDied","Data":"493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66"} Sep 10 00:08:05.628080 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:05.628010 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="493e1cd6df837d27ebfdf02cbce2a5d9a7dcff45201d709871faad4c36a79a66" Sep 10 00:08:13.827151 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.827122 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69"] Sep 10 00:08:13.827548 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.827381 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" containerName="image-pruner" Sep 10 00:08:13.827548 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.827391 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" containerName="image-pruner" Sep 10 00:08:13.827548 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.827448 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="41bbd9ae-02c6-43bb-bf1e-8afaaceb980e" containerName="image-pruner" Sep 10 00:08:13.830442 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.830426 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.833006 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.832983 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 00:08:13.833006 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.832998 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7d4m7\"" Sep 10 00:08:13.833172 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.832995 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 00:08:13.839970 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.839948 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69"] Sep 10 00:08:13.864743 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.864705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.864931 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.864760 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.864931 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.864898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp57v\" (UniqueName: \"kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.965776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.965743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.965776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.965778 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp57v\" (UniqueName: \"kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.966008 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.965833 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.966149 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.966127 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.966207 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.966149 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:13.974647 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:13.974624 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp57v\" (UniqueName: \"kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:14.139535 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:14.139437 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:14.424126 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:14.424050 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69"] Sep 10 00:08:14.427701 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:08:14.427668 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4603017e_9b6b_4475_a181_6f9d41c93f78.slice/crio-8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c WatchSource:0}: Error finding container 8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c: Status 404 returned error can't find the container with id 8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c Sep 10 00:08:14.662765 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:14.662726 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" event={"ID":"4603017e-9b6b-4475-a181-6f9d41c93f78","Type":"ContainerStarted","Data":"8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c"} Sep 10 00:08:16.428965 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.428926 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg"] Sep 10 00:08:16.432204 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.432185 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.434351 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.434330 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 00:08:16.435150 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.435129 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 00:08:16.435250 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.435170 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mcnwn\"" Sep 10 00:08:16.443265 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.443245 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg"] Sep 10 00:08:16.487690 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.487654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzf28\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-kube-api-access-hzf28\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.487847 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.487697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.588922 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.588885 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzf28\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-kube-api-access-hzf28\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.588922 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.588929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.597066 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.597026 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzf28\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-kube-api-access-hzf28\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.597066 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.597048 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99984a72-6bfa-4169-8d69-8eb3c2abd28c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bpqkg\" (UID: \"99984a72-6bfa-4169-8d69-8eb3c2abd28c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:16.751568 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:16.751509 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" Sep 10 00:08:17.030564 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:17.030465 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg"] Sep 10 00:08:17.033578 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:08:17.033465 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99984a72_6bfa_4169_8d69_8eb3c2abd28c.slice/crio-d1cc95c14bf21f62d04424ebade839ae2554949df2ff9e7f511a81f3a5067dca WatchSource:0}: Error finding container d1cc95c14bf21f62d04424ebade839ae2554949df2ff9e7f511a81f3a5067dca: Status 404 returned error can't find the container with id d1cc95c14bf21f62d04424ebade839ae2554949df2ff9e7f511a81f3a5067dca Sep 10 00:08:17.676919 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:17.676881 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" event={"ID":"99984a72-6bfa-4169-8d69-8eb3c2abd28c","Type":"ContainerStarted","Data":"d1cc95c14bf21f62d04424ebade839ae2554949df2ff9e7f511a81f3a5067dca"} Sep 10 00:08:21.696678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:21.696636 2582 generic.go:356] "Generic (PLEG): container finished" podID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerID="12c5af49d51e7d24e9cf6f48f8a10214aa38a5cf9ec814e8f263a7eaeb0c6ee8" exitCode=0 Sep 10 00:08:21.697329 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:21.696757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" event={"ID":"4603017e-9b6b-4475-a181-6f9d41c93f78","Type":"ContainerDied","Data":"12c5af49d51e7d24e9cf6f48f8a10214aa38a5cf9ec814e8f263a7eaeb0c6ee8"} Sep 10 00:08:24.914645 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.914613 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j8cd"] Sep 10 00:08:24.917488 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.917472 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:24.919610 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.919590 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ksxwb\"" Sep 10 00:08:24.926948 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.926926 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j8cd"] Sep 10 00:08:24.964689 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.964652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:24.964689 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:24.964692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjff4\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-kube-api-access-kjff4\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.065788 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.065763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.065906 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.065801 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjff4\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-kube-api-access-kjff4\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.072721 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.072698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.072943 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.072926 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjff4\" (UniqueName: \"kubernetes.io/projected/41e07824-d21b-4e2b-ae7f-4771fb01f0b5-kube-api-access-kjff4\") pod \"cert-manager-55b4d954d5-8j8cd\" (UID: \"41e07824-d21b-4e2b-ae7f-4771fb01f0b5\") " pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.226861 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.226827 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8j8cd" Sep 10 00:08:25.497306 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.497209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j8cd"] Sep 10 00:08:25.499775 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:08:25.499742 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41e07824_d21b_4e2b_ae7f_4771fb01f0b5.slice/crio-367b2d359130d49ba7c7a48427efa16f2145fc5448ecf3e783b6467f360a02e8 WatchSource:0}: Error finding container 367b2d359130d49ba7c7a48427efa16f2145fc5448ecf3e783b6467f360a02e8: Status 404 returned error can't find the container with id 367b2d359130d49ba7c7a48427efa16f2145fc5448ecf3e783b6467f360a02e8 Sep 10 00:08:25.716336 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.716299 2582 generic.go:356] "Generic (PLEG): container finished" podID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerID="dfd6eee300edd25c0e14055d1b26a325074121f5dc88d7d7c3101a4b843860a4" exitCode=0 Sep 10 00:08:25.716541 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.716379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" event={"ID":"4603017e-9b6b-4475-a181-6f9d41c93f78","Type":"ContainerDied","Data":"dfd6eee300edd25c0e14055d1b26a325074121f5dc88d7d7c3101a4b843860a4"} Sep 10 00:08:25.718188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.718165 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8j8cd" event={"ID":"41e07824-d21b-4e2b-ae7f-4771fb01f0b5","Type":"ContainerStarted","Data":"d290b44f84188a40ef629bc870addb6dca02f70de5195a17fc9e20a4be6a74f3"} Sep 10 00:08:25.718284 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.718197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8j8cd" event={"ID":"41e07824-d21b-4e2b-ae7f-4771fb01f0b5","Type":"ContainerStarted","Data":"367b2d359130d49ba7c7a48427efa16f2145fc5448ecf3e783b6467f360a02e8"} Sep 10 00:08:25.719798 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.719776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" event={"ID":"99984a72-6bfa-4169-8d69-8eb3c2abd28c","Type":"ContainerStarted","Data":"1eb4bb10a937ec943f1561686cfbd6fda60a560ba48846a7207b89c28d52ce6e"} Sep 10 00:08:25.745000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.744959 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8j8cd" podStartSLOduration=1.744946511 podStartE2EDuration="1.744946511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:08:25.743127846 +0000 UTC m=+508.424060599" watchObservedRunningTime="2026-09-10 00:08:25.744946511 +0000 UTC m=+508.425879247" Sep 10 00:08:25.759073 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:25.758992 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bpqkg" podStartSLOduration=1.817534705 podStartE2EDuration="9.758976601s" totalImagesPullingTime="7.941441896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:08:25.758223773 +0000 UTC m=+508.439156536" watchObservedRunningTime="2026-09-10 00:08:25.758976601 +0000 UTC m=+508.439909342" Sep 10 00:08:33.758846 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:33.758806 2582 generic.go:356] "Generic (PLEG): container finished" podID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerID="04156a3a209303ad006cf20c71d945baef3f72d41a4f84e17c1f9888dabdf581" exitCode=0 Sep 10 00:08:33.759257 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:33.758861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" event={"ID":"4603017e-9b6b-4475-a181-6f9d41c93f78","Type":"ContainerDied","Data":"04156a3a209303ad006cf20c71d945baef3f72d41a4f84e17c1f9888dabdf581"} Sep 10 00:08:34.885939 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.885910 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:34.954242 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.954203 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp57v\" (UniqueName: \"kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v\") pod \"4603017e-9b6b-4475-a181-6f9d41c93f78\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " Sep 10 00:08:34.954425 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.954307 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle\") pod \"4603017e-9b6b-4475-a181-6f9d41c93f78\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " Sep 10 00:08:34.954425 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.954345 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util\") pod \"4603017e-9b6b-4475-a181-6f9d41c93f78\" (UID: \"4603017e-9b6b-4475-a181-6f9d41c93f78\") " Sep 10 00:08:34.954766 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.954734 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle" (OuterVolumeSpecName: "bundle") pod "4603017e-9b6b-4475-a181-6f9d41c93f78" (UID: "4603017e-9b6b-4475-a181-6f9d41c93f78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:08:34.956627 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.956603 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v" (OuterVolumeSpecName: "kube-api-access-vp57v") pod "4603017e-9b6b-4475-a181-6f9d41c93f78" (UID: "4603017e-9b6b-4475-a181-6f9d41c93f78"). InnerVolumeSpecName "kube-api-access-vp57v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:08:34.960660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:34.960633 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util" (OuterVolumeSpecName: "util") pod "4603017e-9b6b-4475-a181-6f9d41c93f78" (UID: "4603017e-9b6b-4475-a181-6f9d41c93f78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:08:35.055215 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.055144 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:08:35.055215 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.055171 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4603017e-9b6b-4475-a181-6f9d41c93f78-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:08:35.055215 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.055180 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp57v\" (UniqueName: \"kubernetes.io/projected/4603017e-9b6b-4475-a181-6f9d41c93f78-kube-api-access-vp57v\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:08:35.770112 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.770084 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" Sep 10 00:08:35.770281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.770115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhh69" event={"ID":"4603017e-9b6b-4475-a181-6f9d41c93f78","Type":"ContainerDied","Data":"8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c"} Sep 10 00:08:35.770281 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:35.770162 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cd22fbd973099eb40db6e012547f372be8fb7cd6980ea88eaa9fef1cbd44c9c" Sep 10 00:08:54.872692 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872618 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv"] Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872942 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="pull" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872955 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="pull" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872979 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="extract" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872985 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="extract" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.872994 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="util" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.873000 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="util" Sep 10 00:08:54.873114 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.873046 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4603017e-9b6b-4475-a181-6f9d41c93f78" containerName="extract" Sep 10 00:08:54.874865 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.874848 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:54.878785 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878715 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 00:08:54.878785 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878770 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 00:08:54.878785 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878785 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 00:08:54.878997 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878787 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 00:08:54.878997 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878798 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m7xbq\"" Sep 10 00:08:54.878997 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.878874 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 00:08:54.883840 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.883817 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv"] Sep 10 00:08:54.921754 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.921719 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-metrics-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:54.921913 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.921756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-manager-config\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:54.921913 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.921806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7j95\" (UniqueName: \"kubernetes.io/projected/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-kube-api-access-r7j95\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:54.921913 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:54.921847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.022750 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.022706 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-metrics-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.022750 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.022752 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-manager-config\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.023003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.022801 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7j95\" (UniqueName: \"kubernetes.io/projected/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-kube-api-access-r7j95\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.023003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.022826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.023489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.023467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-manager-config\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.025265 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.025235 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-metrics-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.025358 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.025303 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-cert\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.029610 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.029584 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7j95\" (UniqueName: \"kubernetes.io/projected/807d1a33-1a1c-47e1-a6a3-69dbe8b946c5-kube-api-access-r7j95\") pod \"lws-controller-manager-d4c4f85fc-qnqsv\" (UID: \"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5\") " pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.184746 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.184668 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:55.454067 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.453995 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv"] Sep 10 00:08:55.458285 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:08:55.458262 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807d1a33_1a1c_47e1_a6a3_69dbe8b946c5.slice/crio-0531aaa770b110f64ad456e740184e34aaa67c5666ca3efd25c53f3ff2d5a68d WatchSource:0}: Error finding container 0531aaa770b110f64ad456e740184e34aaa67c5666ca3efd25c53f3ff2d5a68d: Status 404 returned error can't find the container with id 0531aaa770b110f64ad456e740184e34aaa67c5666ca3efd25c53f3ff2d5a68d Sep 10 00:08:55.848778 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:55.848742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" event={"ID":"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5","Type":"ContainerStarted","Data":"0531aaa770b110f64ad456e740184e34aaa67c5666ca3efd25c53f3ff2d5a68d"} Sep 10 00:08:57.859888 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:57.859850 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" event={"ID":"807d1a33-1a1c-47e1-a6a3-69dbe8b946c5","Type":"ContainerStarted","Data":"5bca5298a6b164ef4ef0186f4b6b25c62de6ba08600c6c344fd813fa5e63cc99"} Sep 10 00:08:57.860260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:57.860041 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:08:57.879060 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:08:57.879005 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" podStartSLOduration=2.352652262 podStartE2EDuration="3.878990158s" totalImagesPullingTime="1.526337896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:08:57.878172532 +0000 UTC m=+540.559105291" watchObservedRunningTime="2026-09-10 00:08:57.878990158 +0000 UTC m=+540.559922897" Sep 10 00:09:01.257774 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.257735 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt"] Sep 10 00:09:01.261923 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.261905 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.264157 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.264128 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7d4m7\"" Sep 10 00:09:01.264295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.264253 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 00:09:01.265010 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.264926 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 00:09:01.271009 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.270985 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt"] Sep 10 00:09:01.281054 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.281022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q999v\" (UniqueName: \"kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.281199 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.281059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.281199 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.281097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.381973 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.381935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q999v\" (UniqueName: \"kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.381973 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.381978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.382235 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.382013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.382377 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.382360 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.382462 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.382444 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.390939 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.390905 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q999v\" (UniqueName: \"kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.573459 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.573373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:01.868024 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.868000 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt"] Sep 10 00:09:01.870505 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:09:01.870478 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode703bb2a_9740_41a8_9ef3_865ccbcf3866.slice/crio-dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307 WatchSource:0}: Error finding container dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307: Status 404 returned error can't find the container with id dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307 Sep 10 00:09:01.878399 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:01.878373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerStarted","Data":"dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307"} Sep 10 00:09:02.854764 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.854718 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r"] Sep 10 00:09:02.878352 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.878309 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r"] Sep 10 00:09:02.878555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.878432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:02.881508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.881481 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 00:09:02.881676 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.881636 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 00:09:02.881769 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.881599 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 00:09:02.881923 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.881901 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 00:09:02.882034 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.882015 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f56wf\"" Sep 10 00:09:02.887068 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.887035 2582 generic.go:356] "Generic (PLEG): container finished" podID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerID="005da5fbff887cc2939a44fe48d70614f9b36a37e1ecb7dc3d91325aaf7b28a6" exitCode=0 Sep 10 00:09:02.887242 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.887122 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerDied","Data":"005da5fbff887cc2939a44fe48d70614f9b36a37e1ecb7dc3d91325aaf7b28a6"} Sep 10 00:09:02.995002 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.994960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd22b\" (UniqueName: \"kubernetes.io/projected/73d63671-328a-432b-9844-52e214e00c35-kube-api-access-sd22b\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:02.995202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.995030 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:02.995202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.995053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-tmp\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:02.995202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.995080 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-webhook-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:02.995202 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:02.995125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-manifests\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.095688 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.095651 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-webhook-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.095688 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.095690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-manifests\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.095900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.095742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd22b\" (UniqueName: \"kubernetes.io/projected/73d63671-328a-432b-9844-52e214e00c35-kube-api-access-sd22b\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.095951 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.095908 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.096000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.095962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-tmp\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.096241 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.096213 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-manifests\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.098080 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.098056 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d63671-328a-432b-9844-52e214e00c35-tmp\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.098310 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.098291 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-webhook-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.098352 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.098334 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73d63671-328a-432b-9844-52e214e00c35-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.103957 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.103925 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd22b\" (UniqueName: \"kubernetes.io/projected/73d63671-328a-432b-9844-52e214e00c35-kube-api-access-sd22b\") pod \"opendatahub-operator-controller-manager-54f95565dd-g5k6r\" (UID: \"73d63671-328a-432b-9844-52e214e00c35\") " pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.191493 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.191405 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:03.497841 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.497766 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r"] Sep 10 00:09:03.548300 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:09:03.548263 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d63671_328a_432b_9844_52e214e00c35.slice/crio-a8c0116294e28d763b3ef50e785f55dfc474390a9d63d4bf1d6758d442fb8e80 WatchSource:0}: Error finding container a8c0116294e28d763b3ef50e785f55dfc474390a9d63d4bf1d6758d442fb8e80: Status 404 returned error can't find the container with id a8c0116294e28d763b3ef50e785f55dfc474390a9d63d4bf1d6758d442fb8e80 Sep 10 00:09:03.898145 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:03.898106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" event={"ID":"73d63671-328a-432b-9844-52e214e00c35","Type":"ContainerStarted","Data":"a8c0116294e28d763b3ef50e785f55dfc474390a9d63d4bf1d6758d442fb8e80"} Sep 10 00:09:04.906019 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:04.905981 2582 generic.go:356] "Generic (PLEG): container finished" podID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerID="51968870550f22e838ff1c551dcbd486ff71dfa327d71585fcd2d3a6e67e7251" exitCode=0 Sep 10 00:09:04.906497 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:04.906061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerDied","Data":"51968870550f22e838ff1c551dcbd486ff71dfa327d71585fcd2d3a6e67e7251"} Sep 10 00:09:05.914597 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:05.914549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerStarted","Data":"69e3b28c8b6f24dcbda83d9d3d3230d3bbd9b1dbe9d7bbffa242277e68b65105"} Sep 10 00:09:05.936690 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:05.936643 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" podStartSLOduration=3.950127166 podStartE2EDuration="4.936627991s" totalImagesPullingTime="986.500825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:09:05.934325538 +0000 UTC m=+548.615258317" watchObservedRunningTime="2026-09-10 00:09:05.936627991 +0000 UTC m=+548.617560730" Sep 10 00:09:06.920652 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:06.920556 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" event={"ID":"73d63671-328a-432b-9844-52e214e00c35","Type":"ContainerStarted","Data":"dd28396e1492a72835699795969be7ecc8047e4fc34aa30bd2aab151b6212119"} Sep 10 00:09:06.922742 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:06.922717 2582 generic.go:356] "Generic (PLEG): container finished" podID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerID="69e3b28c8b6f24dcbda83d9d3d3230d3bbd9b1dbe9d7bbffa242277e68b65105" exitCode=0 Sep 10 00:09:06.922854 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:06.922756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerDied","Data":"69e3b28c8b6f24dcbda83d9d3d3230d3bbd9b1dbe9d7bbffa242277e68b65105"} Sep 10 00:09:07.928508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:07.928469 2582 generic.go:356] "Generic (PLEG): container finished" podID="73d63671-328a-432b-9844-52e214e00c35" containerID="dd28396e1492a72835699795969be7ecc8047e4fc34aa30bd2aab151b6212119" exitCode=0 Sep 10 00:09:07.928900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:07.928549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" event={"ID":"73d63671-328a-432b-9844-52e214e00c35","Type":"ContainerDied","Data":"dd28396e1492a72835699795969be7ecc8047e4fc34aa30bd2aab151b6212119"} Sep 10 00:09:08.067421 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.067396 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:08.141342 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.141309 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q999v\" (UniqueName: \"kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v\") pod \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " Sep 10 00:09:08.141342 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.141344 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle\") pod \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " Sep 10 00:09:08.141595 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.141373 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util\") pod \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\" (UID: \"e703bb2a-9740-41a8-9ef3-865ccbcf3866\") " Sep 10 00:09:08.142060 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.142037 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle" (OuterVolumeSpecName: "bundle") pod "e703bb2a-9740-41a8-9ef3-865ccbcf3866" (UID: "e703bb2a-9740-41a8-9ef3-865ccbcf3866"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:09:08.143361 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.143339 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v" (OuterVolumeSpecName: "kube-api-access-q999v") pod "e703bb2a-9740-41a8-9ef3-865ccbcf3866" (UID: "e703bb2a-9740-41a8-9ef3-865ccbcf3866"). InnerVolumeSpecName "kube-api-access-q999v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:09:08.145555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.145535 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util" (OuterVolumeSpecName: "util") pod "e703bb2a-9740-41a8-9ef3-865ccbcf3866" (UID: "e703bb2a-9740-41a8-9ef3-865ccbcf3866"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:09:08.242195 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.242166 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:09:08.242295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.242202 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q999v\" (UniqueName: \"kubernetes.io/projected/e703bb2a-9740-41a8-9ef3-865ccbcf3866-kube-api-access-q999v\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:09:08.242295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.242217 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e703bb2a-9740-41a8-9ef3-865ccbcf3866-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:09:08.866433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.866397 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d4c4f85fc-qnqsv" Sep 10 00:09:08.934871 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.934834 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" event={"ID":"73d63671-328a-432b-9844-52e214e00c35","Type":"ContainerStarted","Data":"4e6ea0a64f74a3ae061b022257fe9f74f26a24431abb44e9a6dda4abd256f9c7"} Sep 10 00:09:08.935310 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.934898 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:08.936957 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.936935 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" event={"ID":"e703bb2a-9740-41a8-9ef3-865ccbcf3866","Type":"ContainerDied","Data":"dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307"} Sep 10 00:09:08.936957 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.936947 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0665dt" Sep 10 00:09:08.937103 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.936973 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedf294b85693b105564809b6eae35efdbdff466d4a54475384e2f713b5c7307" Sep 10 00:09:08.984632 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:08.984578 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" podStartSLOduration=2.526618166 podStartE2EDuration="6.984559554s" totalImagesPullingTime="3.403782396s" totalInitContainerRuntime="1.054158992s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:09:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:09:08.95670722 +0000 UTC m=+551.637639978" watchObservedRunningTime="2026-09-10 00:09:08.984559554 +0000 UTC m=+551.665492296" Sep 10 00:09:19.944476 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:19.944444 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54f95565dd-g5k6r" Sep 10 00:09:22.313357 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313323 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d5976c68f-4lwss"] Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313639 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="util" Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313651 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="util" Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313668 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="pull" Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313673 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="pull" Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313683 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="extract" Sep 10 00:09:22.313732 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313689 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="extract" Sep 10 00:09:22.313916 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.313739 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e703bb2a-9740-41a8-9ef3-865ccbcf3866" containerName="extract" Sep 10 00:09:22.316549 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.316533 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.319739 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.319719 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 00:09:22.320081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.320066 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 00:09:22.320195 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.320177 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mjd5n\"" Sep 10 00:09:22.330358 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.330333 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d5976c68f-4lwss"] Sep 10 00:09:22.461106 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.461070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.461282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.461157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tmp\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.461282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.461211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn94v\" (UniqueName: \"kubernetes.io/projected/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-kube-api-access-fn94v\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.562445 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.562405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tmp\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.562627 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.562462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn94v\" (UniqueName: \"kubernetes.io/projected/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-kube-api-access-fn94v\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.562627 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.562491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.562705 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:09:22.562656 2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 10 00:09:22.562738 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:09:22.562729 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs podName:91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c nodeName:}" failed. No retries permitted until 2026-09-10 00:09:23.06270683 +0000 UTC m=+565.743639557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs") pod "kube-auth-proxy-d5976c68f-4lwss" (UID: "91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c") : secret "kube-auth-proxy-tls" not found Sep 10 00:09:22.564805 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.564764 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tmp\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:22.573992 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:22.573964 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn94v\" (UniqueName: \"kubernetes.io/projected/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-kube-api-access-fn94v\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:23.066906 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:23.066866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:23.069422 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:23.069389 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c-tls-certs\") pod \"kube-auth-proxy-d5976c68f-4lwss\" (UID: \"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c\") " pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:23.225817 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:23.225778 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" Sep 10 00:09:23.500484 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:23.500459 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d5976c68f-4lwss"] Sep 10 00:09:23.503108 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:09:23.503074 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fb8eb6_8bdf_450f_b22a_ae82d26e9c2c.slice/crio-fac3e72da7ba011f034f941d51cfcc5cac1c1349e7c3a86e4a377da6629b479e WatchSource:0}: Error finding container fac3e72da7ba011f034f941d51cfcc5cac1c1349e7c3a86e4a377da6629b479e: Status 404 returned error can't find the container with id fac3e72da7ba011f034f941d51cfcc5cac1c1349e7c3a86e4a377da6629b479e Sep 10 00:09:24.011144 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:24.011110 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" event={"ID":"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c","Type":"ContainerStarted","Data":"fac3e72da7ba011f034f941d51cfcc5cac1c1349e7c3a86e4a377da6629b479e"} Sep 10 00:09:27.028839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:27.028798 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" event={"ID":"91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c","Type":"ContainerStarted","Data":"bc6e96a753335bb31762ff93410908fb6d98e40e2b73876da8557c04a42dd9d5"} Sep 10 00:09:27.043921 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:27.043862 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d5976c68f-4lwss" podStartSLOduration=1.926852377 podStartE2EDuration="5.04384667s" totalImagesPullingTime="3.116994293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:09:27.04240254 +0000 UTC m=+569.723335301" watchObservedRunningTime="2026-09-10 00:09:27.04384667 +0000 UTC m=+569.724779412" Sep 10 00:09:57.765675 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:57.765647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:09:57.766633 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:09:57.766615 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:10:07.412861 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.412826 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c8b4555d-dmdq6"] Sep 10 00:10:07.417295 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.417255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.426663 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.426636 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c8b4555d-dmdq6"] Sep 10 00:10:07.533728 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533693 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-oauth-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533728 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf8t9\" (UniqueName: \"kubernetes.io/projected/4352c084-a795-4a87-8dd6-a25948631bea-kube-api-access-pf8t9\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533972 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-oauth-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533972 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533972 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-service-ca\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533972 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533937 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-trusted-ca-bundle\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.533972 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.533961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-console-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635015 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.634961 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-oauth-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635015 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635018 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf8t9\" (UniqueName: \"kubernetes.io/projected/4352c084-a795-4a87-8dd6-a25948631bea-kube-api-access-pf8t9\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635258 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-oauth-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635258 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635258 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-service-ca\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635258 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-trusted-ca-bundle\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635258 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635182 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-console-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635789 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635764 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-oauth-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.635984 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.635965 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-service-ca\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.636063 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.636040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-console-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.636313 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.636289 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4352c084-a795-4a87-8dd6-a25948631bea-trusted-ca-bundle\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.637684 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.637662 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-serving-cert\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.637781 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.637760 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4352c084-a795-4a87-8dd6-a25948631bea-console-oauth-config\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.642336 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.642318 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf8t9\" (UniqueName: \"kubernetes.io/projected/4352c084-a795-4a87-8dd6-a25948631bea-kube-api-access-pf8t9\") pod \"console-55c8b4555d-dmdq6\" (UID: \"4352c084-a795-4a87-8dd6-a25948631bea\") " pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.731921 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.731891 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:07.999939 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:07.999845 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c8b4555d-dmdq6"] Sep 10 00:10:08.002721 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:10:08.002691 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4352c084_a795_4a87_8dd6_a25948631bea.slice/crio-5edebbcfb0dc9b1d0fd086fb57d7bf75f8c0753068cbbb039a1a374feb50e136 WatchSource:0}: Error finding container 5edebbcfb0dc9b1d0fd086fb57d7bf75f8c0753068cbbb039a1a374feb50e136: Status 404 returned error can't find the container with id 5edebbcfb0dc9b1d0fd086fb57d7bf75f8c0753068cbbb039a1a374feb50e136 Sep 10 00:10:08.216697 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:08.216665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c8b4555d-dmdq6" event={"ID":"4352c084-a795-4a87-8dd6-a25948631bea","Type":"ContainerStarted","Data":"c8142e9aa56dc2e9fc5af4c141a5df5a56f4842053df8285ca557d77a61d4161"} Sep 10 00:10:08.216860 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:08.216714 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c8b4555d-dmdq6" event={"ID":"4352c084-a795-4a87-8dd6-a25948631bea","Type":"ContainerStarted","Data":"5edebbcfb0dc9b1d0fd086fb57d7bf75f8c0753068cbbb039a1a374feb50e136"} Sep 10 00:10:08.232417 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:08.232367 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c8b4555d-dmdq6" podStartSLOduration=1.2323479800000001 podStartE2EDuration="1.23234798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:10:08.230904997 +0000 UTC m=+610.911837765" watchObservedRunningTime="2026-09-10 00:10:08.23234798 +0000 UTC m=+610.913280722" Sep 10 00:10:17.732722 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:17.732682 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:17.733261 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:17.732770 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:17.737552 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:17.737508 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:18.267993 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:18.267966 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c8b4555d-dmdq6" Sep 10 00:10:18.309126 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:18.309093 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:10:37.393925 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.393881 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd"] Sep 10 00:10:37.397613 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.397589 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.399873 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.399851 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 00:10:37.400629 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.400610 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7d4m7\"" Sep 10 00:10:37.400629 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.400621 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 00:10:37.406646 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.406621 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd"] Sep 10 00:10:37.492571 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.492538 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7"] Sep 10 00:10:37.495896 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.495879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.498255 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.498234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.498299 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.498279 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d8lf\" (UniqueName: \"kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.498378 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.498361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.502619 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.502589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7"] Sep 10 00:10:37.592628 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.592596 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk"] Sep 10 00:10:37.595993 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.595976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.598851 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.598827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.598948 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.598866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2vdx\" (UniqueName: \"kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.598995 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.598943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.598995 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.598983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8lf\" (UniqueName: \"kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.599112 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.599097 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.599151 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.599133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.599427 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.599408 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.599493 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.599448 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.603873 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.603848 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk"] Sep 10 00:10:37.606546 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.606505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8lf\" (UniqueName: \"kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.693804 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.693726 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf"] Sep 10 00:10:37.697192 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.697176 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.699559 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.699683 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699581 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.699683 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.699774 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n8gp\" (UniqueName: \"kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.699774 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.699857 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699773 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2vdx\" (UniqueName: \"kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.699857 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.699783 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.700095 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.700074 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.705433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.705408 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf"] Sep 10 00:10:37.707282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.707261 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2vdx\" (UniqueName: \"kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.708301 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.708283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:37.801234 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801200 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.801383 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801257 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.801383 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n8gp\" (UniqueName: \"kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.801383 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801322 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.801489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.801489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801436 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgdtm\" (UniqueName: \"kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.801616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801595 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.801759 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.801743 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.806144 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.806115 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:37.808538 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.808493 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n8gp\" (UniqueName: \"kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.902349 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.902316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.902505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.902365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.902505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.902403 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgdtm\" (UniqueName: \"kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.902691 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.902673 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.902793 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.902772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.905479 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.905461 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:37.909861 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.909842 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgdtm\" (UniqueName: \"kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:37.983162 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:37.983137 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd"] Sep 10 00:10:37.985269 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:10:37.985239 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4151a8df_c96c_41b8_916e_5c23b2b945fe.slice/crio-d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b WatchSource:0}: Error finding container d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b: Status 404 returned error can't find the container with id d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b Sep 10 00:10:38.007613 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.007590 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:38.084244 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.084216 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7"] Sep 10 00:10:38.145350 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:10:38.145318 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08fad16_252f_4bf6_9afd_75b82cac90cd.slice/crio-61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c WatchSource:0}: Error finding container 61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c: Status 404 returned error can't find the container with id 61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c Sep 10 00:10:38.177152 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.176719 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk"] Sep 10 00:10:38.178543 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:10:38.178486 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb5ad0d_6567_46c9_b1cc_a15a989425d0.slice/crio-04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e WatchSource:0}: Error finding container 04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e: Status 404 returned error can't find the container with id 04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e Sep 10 00:10:38.281408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.281373 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf"] Sep 10 00:10:38.342280 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:10:38.342242 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d146cd6_390d_49ad_9d97_b758dd32aa19.slice/crio-65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23 WatchSource:0}: Error finding container 65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23: Status 404 returned error can't find the container with id 65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23 Sep 10 00:10:38.357028 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.357000 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" event={"ID":"5d146cd6-390d-49ad-9d97-b758dd32aa19","Type":"ContainerStarted","Data":"65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23"} Sep 10 00:10:38.358501 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.358478 2582 generic.go:356] "Generic (PLEG): container finished" podID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerID="bca69b40b8cc3bd754dd8627dc25f94d3f52a421274964e9959b762d6b2d39ca" exitCode=0 Sep 10 00:10:38.358622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.358558 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" event={"ID":"4151a8df-c96c-41b8-916e-5c23b2b945fe","Type":"ContainerDied","Data":"bca69b40b8cc3bd754dd8627dc25f94d3f52a421274964e9959b762d6b2d39ca"} Sep 10 00:10:38.358622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.358603 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" event={"ID":"4151a8df-c96c-41b8-916e-5c23b2b945fe","Type":"ContainerStarted","Data":"d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b"} Sep 10 00:10:38.359600 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.359582 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 00:10:38.360349 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.360327 2582 generic.go:356] "Generic (PLEG): container finished" podID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerID="1910fe859a7cb5a29d3e1655a1b444765228f9fc3ee0ef21a4bba16f42f76874" exitCode=0 Sep 10 00:10:38.360440 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.360420 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" event={"ID":"d08fad16-252f-4bf6-9afd-75b82cac90cd","Type":"ContainerDied","Data":"1910fe859a7cb5a29d3e1655a1b444765228f9fc3ee0ef21a4bba16f42f76874"} Sep 10 00:10:38.360508 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.360456 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" event={"ID":"d08fad16-252f-4bf6-9afd-75b82cac90cd","Type":"ContainerStarted","Data":"61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c"} Sep 10 00:10:38.362390 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.362366 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerID="105125a2a27145e29a8f6b8cbf9473cd1bc54537d28c05fad29d49bf4c72c3ee" exitCode=0 Sep 10 00:10:38.362480 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.362405 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" event={"ID":"4eb5ad0d-6567-46c9-b1cc-a15a989425d0","Type":"ContainerDied","Data":"105125a2a27145e29a8f6b8cbf9473cd1bc54537d28c05fad29d49bf4c72c3ee"} Sep 10 00:10:38.362480 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:38.362437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" event={"ID":"4eb5ad0d-6567-46c9-b1cc-a15a989425d0","Type":"ContainerStarted","Data":"04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e"} Sep 10 00:10:39.369461 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:39.369424 2582 generic.go:356] "Generic (PLEG): container finished" podID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerID="d0bf7f18a87fb09ca887a7365d1fdbce2359ccab396e37fe0976c5811df12731" exitCode=0 Sep 10 00:10:39.369771 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:39.369484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" event={"ID":"5d146cd6-390d-49ad-9d97-b758dd32aa19","Type":"ContainerDied","Data":"d0bf7f18a87fb09ca887a7365d1fdbce2359ccab396e37fe0976c5811df12731"} Sep 10 00:10:40.379549 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.379493 2582 generic.go:356] "Generic (PLEG): container finished" podID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerID="b125156efcc0defcc1c8262d75f95f435332b61edefdc99aca905931ff472456" exitCode=0 Sep 10 00:10:40.379989 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.379553 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" event={"ID":"4151a8df-c96c-41b8-916e-5c23b2b945fe","Type":"ContainerDied","Data":"b125156efcc0defcc1c8262d75f95f435332b61edefdc99aca905931ff472456"} Sep 10 00:10:40.381368 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.381341 2582 generic.go:356] "Generic (PLEG): container finished" podID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerID="d42a536c65f0426bff8b16b1c7705a7f0eae25e484d209fb3685372b1d7a481b" exitCode=0 Sep 10 00:10:40.381458 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.381368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" event={"ID":"d08fad16-252f-4bf6-9afd-75b82cac90cd","Type":"ContainerDied","Data":"d42a536c65f0426bff8b16b1c7705a7f0eae25e484d209fb3685372b1d7a481b"} Sep 10 00:10:40.383333 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.383312 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerID="7d823252d5a7f1311610edbe888397df53f860a878c689ec3aa62d9936105dc9" exitCode=0 Sep 10 00:10:40.383412 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:40.383352 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" event={"ID":"4eb5ad0d-6567-46c9-b1cc-a15a989425d0","Type":"ContainerDied","Data":"7d823252d5a7f1311610edbe888397df53f860a878c689ec3aa62d9936105dc9"} Sep 10 00:10:41.393017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.392980 2582 generic.go:356] "Generic (PLEG): container finished" podID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerID="78e852569b66b55233d222f56a783d730acab4201ff7a7bd68322c335b76814d" exitCode=0 Sep 10 00:10:41.393429 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.393064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" event={"ID":"4151a8df-c96c-41b8-916e-5c23b2b945fe","Type":"ContainerDied","Data":"78e852569b66b55233d222f56a783d730acab4201ff7a7bd68322c335b76814d"} Sep 10 00:10:41.395185 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.395164 2582 generic.go:356] "Generic (PLEG): container finished" podID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerID="073f7b7d922905b55e5039b999a2e982cff88a9bf8cda6dd0c038bbd79df79f3" exitCode=0 Sep 10 00:10:41.395303 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.395236 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" event={"ID":"d08fad16-252f-4bf6-9afd-75b82cac90cd","Type":"ContainerDied","Data":"073f7b7d922905b55e5039b999a2e982cff88a9bf8cda6dd0c038bbd79df79f3"} Sep 10 00:10:41.397348 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.397329 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerID="a60c65291516ed728778f723fe905bef02902739e41dfa490c9c4c6c79b0bd2c" exitCode=0 Sep 10 00:10:41.397450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.397408 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" event={"ID":"4eb5ad0d-6567-46c9-b1cc-a15a989425d0","Type":"ContainerDied","Data":"a60c65291516ed728778f723fe905bef02902739e41dfa490c9c4c6c79b0bd2c"} Sep 10 00:10:41.398943 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.398925 2582 generic.go:356] "Generic (PLEG): container finished" podID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerID="73aedf3595355dfe55fcaefb1013450db285a48c8035444e1e2787b898cface2" exitCode=0 Sep 10 00:10:41.399018 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:41.398972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" event={"ID":"5d146cd6-390d-49ad-9d97-b758dd32aa19","Type":"ContainerDied","Data":"73aedf3595355dfe55fcaefb1013450db285a48c8035444e1e2787b898cface2"} Sep 10 00:10:42.406297 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.406261 2582 generic.go:356] "Generic (PLEG): container finished" podID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerID="e9134d84930e64827977ba9782874fbcfbf4ded5e9f0c9e37f8414b2019203ea" exitCode=0 Sep 10 00:10:42.406812 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.406352 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" event={"ID":"5d146cd6-390d-49ad-9d97-b758dd32aa19","Type":"ContainerDied","Data":"e9134d84930e64827977ba9782874fbcfbf4ded5e9f0c9e37f8414b2019203ea"} Sep 10 00:10:42.571611 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.571583 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:42.622332 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.622301 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:42.625264 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.625240 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:42.744332 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744286 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util\") pod \"d08fad16-252f-4bf6-9afd-75b82cac90cd\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n8gp\" (UniqueName: \"kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp\") pod \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744409 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d8lf\" (UniqueName: \"kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf\") pod \"4151a8df-c96c-41b8-916e-5c23b2b945fe\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744438 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util\") pod \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744465 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle\") pod \"4151a8df-c96c-41b8-916e-5c23b2b945fe\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744490 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2vdx\" (UniqueName: \"kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx\") pod \"d08fad16-252f-4bf6-9afd-75b82cac90cd\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744531 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle\") pod \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\" (UID: \"4eb5ad0d-6567-46c9-b1cc-a15a989425d0\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744563 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle\") pod \"d08fad16-252f-4bf6-9afd-75b82cac90cd\" (UID: \"d08fad16-252f-4bf6-9afd-75b82cac90cd\") " Sep 10 00:10:42.744604 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.744605 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util\") pod \"4151a8df-c96c-41b8-916e-5c23b2b945fe\" (UID: \"4151a8df-c96c-41b8-916e-5c23b2b945fe\") " Sep 10 00:10:42.745399 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.745363 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle" (OuterVolumeSpecName: "bundle") pod "4151a8df-c96c-41b8-916e-5c23b2b945fe" (UID: "4151a8df-c96c-41b8-916e-5c23b2b945fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.745817 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.745785 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle" (OuterVolumeSpecName: "bundle") pod "4eb5ad0d-6567-46c9-b1cc-a15a989425d0" (UID: "4eb5ad0d-6567-46c9-b1cc-a15a989425d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.745954 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.745932 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle" (OuterVolumeSpecName: "bundle") pod "d08fad16-252f-4bf6-9afd-75b82cac90cd" (UID: "d08fad16-252f-4bf6-9afd-75b82cac90cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.746776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.746741 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp" (OuterVolumeSpecName: "kube-api-access-6n8gp") pod "4eb5ad0d-6567-46c9-b1cc-a15a989425d0" (UID: "4eb5ad0d-6567-46c9-b1cc-a15a989425d0"). InnerVolumeSpecName "kube-api-access-6n8gp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:10:42.747090 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.747059 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf" (OuterVolumeSpecName: "kube-api-access-2d8lf") pod "4151a8df-c96c-41b8-916e-5c23b2b945fe" (UID: "4151a8df-c96c-41b8-916e-5c23b2b945fe"). InnerVolumeSpecName "kube-api-access-2d8lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:10:42.747742 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.747720 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx" (OuterVolumeSpecName: "kube-api-access-f2vdx") pod "d08fad16-252f-4bf6-9afd-75b82cac90cd" (UID: "d08fad16-252f-4bf6-9afd-75b82cac90cd"). InnerVolumeSpecName "kube-api-access-f2vdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:10:42.750851 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.750824 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util" (OuterVolumeSpecName: "util") pod "4eb5ad0d-6567-46c9-b1cc-a15a989425d0" (UID: "4eb5ad0d-6567-46c9-b1cc-a15a989425d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.753330 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.753308 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util" (OuterVolumeSpecName: "util") pod "d08fad16-252f-4bf6-9afd-75b82cac90cd" (UID: "d08fad16-252f-4bf6-9afd-75b82cac90cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.754128 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.754110 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util" (OuterVolumeSpecName: "util") pod "4151a8df-c96c-41b8-916e-5c23b2b945fe" (UID: "4151a8df-c96c-41b8-916e-5c23b2b945fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:42.846223 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846187 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846223 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846218 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n8gp\" (UniqueName: \"kubernetes.io/projected/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-kube-api-access-6n8gp\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846223 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846230 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d8lf\" (UniqueName: \"kubernetes.io/projected/4151a8df-c96c-41b8-916e-5c23b2b945fe-kube-api-access-2d8lf\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846240 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846249 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846257 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2vdx\" (UniqueName: \"kubernetes.io/projected/d08fad16-252f-4bf6-9afd-75b82cac90cd-kube-api-access-f2vdx\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846265 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eb5ad0d-6567-46c9-b1cc-a15a989425d0-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846273 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d08fad16-252f-4bf6-9afd-75b82cac90cd-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:42.846450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:42.846279 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4151a8df-c96c-41b8-916e-5c23b2b945fe-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.332226 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.332184 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84dc48bd5-tn2b2" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerName="console" containerID="cri-o://8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f" gracePeriod=15 Sep 10 00:10:43.389389 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.389346 2582 patch_prober.go:28] interesting pod/console-84dc48bd5-tn2b2 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" start-of-body= Sep 10 00:10:43.389560 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.389403 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-84dc48bd5-tn2b2" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerName="console" probeResult="failure" output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" Sep 10 00:10:43.413555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.413511 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" Sep 10 00:10:43.413555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.413542 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7zgvd" event={"ID":"4151a8df-c96c-41b8-916e-5c23b2b945fe","Type":"ContainerDied","Data":"d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b"} Sep 10 00:10:43.414053 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.413595 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d30513c79912f4decfe8ae3b25b1728c57257e1134ae05e73e0305e09765f46b" Sep 10 00:10:43.415462 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.415440 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" Sep 10 00:10:43.415615 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.415465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdkw7" event={"ID":"d08fad16-252f-4bf6-9afd-75b82cac90cd","Type":"ContainerDied","Data":"61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c"} Sep 10 00:10:43.415615 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.415505 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61e83392bee35ab3318027dabd9db1206fff940d44905690eabb8c9562f1761c" Sep 10 00:10:43.417500 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.417481 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" Sep 10 00:10:43.417500 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.417486 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847anblqk" event={"ID":"4eb5ad0d-6567-46c9-b1cc-a15a989425d0","Type":"ContainerDied","Data":"04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e"} Sep 10 00:10:43.417728 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.417535 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04f7e8aa582a8569e76776b8567d85c3e974a2fff5bcded1971a1a49743a269e" Sep 10 00:10:43.539188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.539161 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:43.592462 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.592407 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:10:43.653103 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.653069 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util\") pod \"5d146cd6-390d-49ad-9d97-b758dd32aa19\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " Sep 10 00:10:43.653263 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.653182 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgdtm\" (UniqueName: \"kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm\") pod \"5d146cd6-390d-49ad-9d97-b758dd32aa19\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " Sep 10 00:10:43.653263 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.653222 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle\") pod \"5d146cd6-390d-49ad-9d97-b758dd32aa19\" (UID: \"5d146cd6-390d-49ad-9d97-b758dd32aa19\") " Sep 10 00:10:43.653685 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.653655 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle" (OuterVolumeSpecName: "bundle") pod "5d146cd6-390d-49ad-9d97-b758dd32aa19" (UID: "5d146cd6-390d-49ad-9d97-b758dd32aa19"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:43.655620 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.655595 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm" (OuterVolumeSpecName: "kube-api-access-hgdtm") pod "5d146cd6-390d-49ad-9d97-b758dd32aa19" (UID: "5d146cd6-390d-49ad-9d97-b758dd32aa19"). InnerVolumeSpecName "kube-api-access-hgdtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:10:43.674600 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.674567 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util" (OuterVolumeSpecName: "util") pod "5d146cd6-390d-49ad-9d97-b758dd32aa19" (UID: "5d146cd6-390d-49ad-9d97-b758dd32aa19"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:10:43.754622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754586 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754622 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754624 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754688 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754713 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754743 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754786 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvt7g\" (UniqueName: \"kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.754837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.754835 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert\") pod \"8bc623d6-5ec0-4483-8617-25edf1fce050\" (UID: \"8bc623d6-5ec0-4483-8617-25edf1fce050\") " Sep 10 00:10:43.755115 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755096 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgdtm\" (UniqueName: \"kubernetes.io/projected/5d146cd6-390d-49ad-9d97-b758dd32aa19-kube-api-access-hgdtm\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.755179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755105 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:10:43.755179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755121 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.755179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755151 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config" (OuterVolumeSpecName: "console-config") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:10:43.755179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755164 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:10:43.755370 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755160 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d146cd6-390d-49ad-9d97-b758dd32aa19-util\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.755481 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.755376 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:10:43.756953 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.756932 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:10:43.757028 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.756993 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:10:43.757028 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.757007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g" (OuterVolumeSpecName: "kube-api-access-wvt7g") pod "8bc623d6-5ec0-4483-8617-25edf1fce050" (UID: "8bc623d6-5ec0-4483-8617-25edf1fce050"). InnerVolumeSpecName "kube-api-access-wvt7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856345 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-oauth-serving-cert\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856374 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-service-ca\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856384 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-trusted-ca-bundle\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856392 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-oauth-config\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856401 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bc623d6-5ec0-4483-8617-25edf1fce050-console-config\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856408 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856409 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bc623d6-5ec0-4483-8617-25edf1fce050-console-serving-cert\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:43.856689 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:43.856417 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvt7g\" (UniqueName: \"kubernetes.io/projected/8bc623d6-5ec0-4483-8617-25edf1fce050-kube-api-access-wvt7g\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:10:44.424028 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.423993 2582 generic.go:356] "Generic (PLEG): container finished" podID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerID="8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f" exitCode=0 Sep 10 00:10:44.424499 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.424034 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dc48bd5-tn2b2" Sep 10 00:10:44.424499 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.424082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dc48bd5-tn2b2" event={"ID":"8bc623d6-5ec0-4483-8617-25edf1fce050","Type":"ContainerDied","Data":"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f"} Sep 10 00:10:44.424499 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.424138 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dc48bd5-tn2b2" event={"ID":"8bc623d6-5ec0-4483-8617-25edf1fce050","Type":"ContainerDied","Data":"444690284d49e56cf206006eeaef89e5cf71d5bdf09a3cd8f42a5a339943eb4a"} Sep 10 00:10:44.424499 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.424169 2582 scope.go:117] "RemoveContainer" containerID="8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f" Sep 10 00:10:44.426323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.426300 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" event={"ID":"5d146cd6-390d-49ad-9d97-b758dd32aa19","Type":"ContainerDied","Data":"65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23"} Sep 10 00:10:44.426323 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.426315 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k5kzf" Sep 10 00:10:44.426484 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.426342 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65c352900f9ec8afd69183969835075c50a260f35322809146e5fb29d2a39f23" Sep 10 00:10:44.433340 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.433325 2582 scope.go:117] "RemoveContainer" containerID="8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f" Sep 10 00:10:44.433656 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:10:44.433636 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f\": container with ID starting with 8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f not found: ID does not exist" containerID="8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f" Sep 10 00:10:44.433703 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.433664 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f"} err="failed to get container status \"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f\": rpc error: code = NotFound desc = could not find container \"8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f\": container with ID starting with 8dbd3193b949b8231420800d19f3c81e89f5d8e3e20dadda5ff4e4b361e81a8f not found: ID does not exist" Sep 10 00:10:44.441777 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.441756 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:10:44.445327 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:44.445308 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84dc48bd5-tn2b2"] Sep 10 00:10:45.827890 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:10:45.827851 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" path="/var/lib/kubelet/pods/8bc623d6-5ec0-4483-8617-25edf1fce050/volumes" Sep 10 00:11:12.063927 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.063890 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9"] Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064239 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064249 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064259 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064264 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064273 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064279 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064284 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064289 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064296 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064301 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064310 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064315 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064324 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064329 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="pull" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064339 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064344 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064350 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerName="console" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064355 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerName="console" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064365 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064369 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064376 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064380 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064385 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064390 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="util" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064395 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064399 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064445 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08fad16-252f-4bf6-9afd-75b82cac90cd" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064453 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc623d6-5ec0-4483-8617-25edf1fce050" containerName="console" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064460 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb5ad0d-6567-46c9-b1cc-a15a989425d0" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064468 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4151a8df-c96c-41b8-916e-5c23b2b945fe" containerName="extract" Sep 10 00:11:12.064557 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.064474 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d146cd6-390d-49ad-9d97-b758dd32aa19" containerName="extract" Sep 10 00:11:12.069278 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.069255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:12.071898 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.071873 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 00:11:12.072559 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.072539 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 00:11:12.072640 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.072541 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m4j9t\"" Sep 10 00:11:12.080116 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.080090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9"] Sep 10 00:11:12.181694 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.181657 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kgn5\" (UniqueName: \"kubernetes.io/projected/02653c5c-3d42-456b-bc77-df7fd7397a00-kube-api-access-4kgn5\") pod \"limitador-operator-controller-manager-74495c69f8-j9vn9\" (UID: \"02653c5c-3d42-456b-bc77-df7fd7397a00\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:12.282072 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.282027 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kgn5\" (UniqueName: \"kubernetes.io/projected/02653c5c-3d42-456b-bc77-df7fd7397a00-kube-api-access-4kgn5\") pod \"limitador-operator-controller-manager-74495c69f8-j9vn9\" (UID: \"02653c5c-3d42-456b-bc77-df7fd7397a00\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:12.292779 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.292730 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kgn5\" (UniqueName: \"kubernetes.io/projected/02653c5c-3d42-456b-bc77-df7fd7397a00-kube-api-access-4kgn5\") pod \"limitador-operator-controller-manager-74495c69f8-j9vn9\" (UID: \"02653c5c-3d42-456b-bc77-df7fd7397a00\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:12.381336 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.381243 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:12.656297 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:12.656216 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9"] Sep 10 00:11:12.659817 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:11:12.659787 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02653c5c_3d42_456b_bc77_df7fd7397a00.slice/crio-a1a68a7ea51e066c1f5870469b266ed43f63ca43148eadb2c5a8d0be4cc2143e WatchSource:0}: Error finding container a1a68a7ea51e066c1f5870469b266ed43f63ca43148eadb2c5a8d0be4cc2143e: Status 404 returned error can't find the container with id a1a68a7ea51e066c1f5870469b266ed43f63ca43148eadb2c5a8d0be4cc2143e Sep 10 00:11:13.568446 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:13.568414 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" event={"ID":"02653c5c-3d42-456b-bc77-df7fd7397a00","Type":"ContainerStarted","Data":"a1a68a7ea51e066c1f5870469b266ed43f63ca43148eadb2c5a8d0be4cc2143e"} Sep 10 00:11:16.591753 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:16.591717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" event={"ID":"02653c5c-3d42-456b-bc77-df7fd7397a00","Type":"ContainerStarted","Data":"214a8ccfdfe5160ae4d41cc956e7d21c0ad61397778b4aa02cdec5acd6a280ca"} Sep 10 00:11:16.592147 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:16.591792 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:16.609828 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:16.609766 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" podStartSLOduration=1.737475537 podStartE2EDuration="4.609750437s" totalImagesPullingTime="2.8722749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:11:16.608414542 +0000 UTC m=+679.289347319" watchObservedRunningTime="2026-09-10 00:11:16.609750437 +0000 UTC m=+679.290683178" Sep 10 00:11:27.598677 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:27.598646 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-j9vn9" Sep 10 00:11:56.245643 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.245560 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:11:56.249267 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.249242 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:11:56.251628 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.251604 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zpzlp\"" Sep 10 00:11:56.256176 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.256122 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:11:56.256306 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.256261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jmn\" (UniqueName: \"kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn\") pod \"authorino-7bcfd8f9cd-nwbb6\" (UID: \"e1dd0e11-f30a-4c79-a9f3-b954e27374ec\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:11:56.356905 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.356871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jmn\" (UniqueName: \"kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn\") pod \"authorino-7bcfd8f9cd-nwbb6\" (UID: \"e1dd0e11-f30a-4c79-a9f3-b954e27374ec\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:11:56.364457 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.364425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jmn\" (UniqueName: \"kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn\") pod \"authorino-7bcfd8f9cd-nwbb6\" (UID: \"e1dd0e11-f30a-4c79-a9f3-b954e27374ec\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:11:56.425022 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.424992 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:11:56.428327 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.428310 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:11:56.437887 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.437841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:11:56.457283 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.457253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmqkw\" (UniqueName: \"kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw\") pod \"authorino-59c974bd48-m6bkt\" (UID: \"9d8fa327-5d97-4c6c-8e49-29513e1fe918\") " pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:11:56.558753 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.558670 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmqkw\" (UniqueName: \"kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw\") pod \"authorino-59c974bd48-m6bkt\" (UID: \"9d8fa327-5d97-4c6c-8e49-29513e1fe918\") " pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:11:56.559489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.559470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:11:56.566237 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.566211 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmqkw\" (UniqueName: \"kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw\") pod \"authorino-59c974bd48-m6bkt\" (UID: \"9d8fa327-5d97-4c6c-8e49-29513e1fe918\") " pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:11:56.737862 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.737824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:11:56.828677 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:56.828651 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:11:56.830774 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:11:56.830751 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1dd0e11_f30a_4c79_a9f3_b954e27374ec.slice/crio-7dac4cd40729f1788198e22f47d0dac1464d685d4571430dec87f6be707dfd60 WatchSource:0}: Error finding container 7dac4cd40729f1788198e22f47d0dac1464d685d4571430dec87f6be707dfd60: Status 404 returned error can't find the container with id 7dac4cd40729f1788198e22f47d0dac1464d685d4571430dec87f6be707dfd60 Sep 10 00:11:57.008497 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:57.008463 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:11:57.011464 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:11:57.011436 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d8fa327_5d97_4c6c_8e49_29513e1fe918.slice/crio-a43d26d156a3837a54182ac1e0a6589fdbcc89824fda1bde58484b64e601057f WatchSource:0}: Error finding container a43d26d156a3837a54182ac1e0a6589fdbcc89824fda1bde58484b64e601057f: Status 404 returned error can't find the container with id a43d26d156a3837a54182ac1e0a6589fdbcc89824fda1bde58484b64e601057f Sep 10 00:11:57.804582 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:57.804541 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-m6bkt" event={"ID":"9d8fa327-5d97-4c6c-8e49-29513e1fe918","Type":"ContainerStarted","Data":"a43d26d156a3837a54182ac1e0a6589fdbcc89824fda1bde58484b64e601057f"} Sep 10 00:11:57.805946 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:57.805919 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" event={"ID":"e1dd0e11-f30a-4c79-a9f3-b954e27374ec","Type":"ContainerStarted","Data":"7dac4cd40729f1788198e22f47d0dac1464d685d4571430dec87f6be707dfd60"} Sep 10 00:11:59.818958 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:59.818914 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" event={"ID":"e1dd0e11-f30a-4c79-a9f3-b954e27374ec","Type":"ContainerStarted","Data":"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471"} Sep 10 00:11:59.820328 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:59.820303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-m6bkt" event={"ID":"9d8fa327-5d97-4c6c-8e49-29513e1fe918","Type":"ContainerStarted","Data":"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c"} Sep 10 00:11:59.831608 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:59.831567 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" podStartSLOduration=1.650887083 podStartE2EDuration="3.831554782s" totalImagesPullingTime="2.180667699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:11:59.831053344 +0000 UTC m=+722.511986087" watchObservedRunningTime="2026-09-10 00:11:59.831554782 +0000 UTC m=+722.512487520" Sep 10 00:11:59.846950 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:59.846895 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-m6bkt" podStartSLOduration=1.841750363 podStartE2EDuration="3.846875888s" totalImagesPullingTime="2.005125525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:11:59.84491644 +0000 UTC m=+722.525849184" watchObservedRunningTime="2026-09-10 00:11:59.846875888 +0000 UTC m=+722.527808631" Sep 10 00:11:59.867729 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:11:59.867693 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:12:01.830082 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:01.830038 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" podUID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" containerName="authorino" containerID="cri-o://8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471" gracePeriod=30 Sep 10 00:12:02.073974 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.073945 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:12:02.107975 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.107891 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86jmn\" (UniqueName: \"kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn\") pod \"e1dd0e11-f30a-4c79-a9f3-b954e27374ec\" (UID: \"e1dd0e11-f30a-4c79-a9f3-b954e27374ec\") " Sep 10 00:12:02.110119 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.110096 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn" (OuterVolumeSpecName: "kube-api-access-86jmn") pod "e1dd0e11-f30a-4c79-a9f3-b954e27374ec" (UID: "e1dd0e11-f30a-4c79-a9f3-b954e27374ec"). InnerVolumeSpecName "kube-api-access-86jmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:12:02.209097 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.209055 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86jmn\" (UniqueName: \"kubernetes.io/projected/e1dd0e11-f30a-4c79-a9f3-b954e27374ec-kube-api-access-86jmn\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:12:02.836060 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.836026 2582 generic.go:356] "Generic (PLEG): container finished" podID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" containerID="8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471" exitCode=0 Sep 10 00:12:02.836547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.836065 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" Sep 10 00:12:02.836547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.836112 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" event={"ID":"e1dd0e11-f30a-4c79-a9f3-b954e27374ec","Type":"ContainerDied","Data":"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471"} Sep 10 00:12:02.836547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.836156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nwbb6" event={"ID":"e1dd0e11-f30a-4c79-a9f3-b954e27374ec","Type":"ContainerDied","Data":"7dac4cd40729f1788198e22f47d0dac1464d685d4571430dec87f6be707dfd60"} Sep 10 00:12:02.836547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.836173 2582 scope.go:117] "RemoveContainer" containerID="8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471" Sep 10 00:12:02.845555 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.845510 2582 scope.go:117] "RemoveContainer" containerID="8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471" Sep 10 00:12:02.845860 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:12:02.845840 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471\": container with ID starting with 8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471 not found: ID does not exist" containerID="8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471" Sep 10 00:12:02.845912 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.845868 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471"} err="failed to get container status \"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471\": rpc error: code = NotFound desc = could not find container \"8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471\": container with ID starting with 8f725f23f3fa6c4ffb2f4e0e7448aab8d982551e802d5cf5cdfa58d1e3a27471 not found: ID does not exist" Sep 10 00:12:02.856432 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.856402 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:12:02.858272 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:02.858250 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nwbb6"] Sep 10 00:12:03.827457 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:03.827425 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" path="/var/lib/kubelet/pods/e1dd0e11-f30a-4c79-a9f3-b954e27374ec/volumes" Sep 10 00:12:25.305831 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.305793 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-9m5rm"] Sep 10 00:12:25.306331 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.306310 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" containerName="authorino" Sep 10 00:12:25.306426 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.306334 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" containerName="authorino" Sep 10 00:12:25.306479 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.306454 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dd0e11-f30a-4c79-a9f3-b954e27374ec" containerName="authorino" Sep 10 00:12:25.309303 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.309283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:25.314765 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.314741 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-9m5rm"] Sep 10 00:12:25.423023 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.422992 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flthn\" (UniqueName: \"kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn\") pod \"authorino-557f876f69-9m5rm\" (UID: \"15ad29a0-1878-44e9-bc36-819b294b7e61\") " pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:25.521658 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.521622 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9m5rm"] Sep 10 00:12:25.521885 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:12:25.521868 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-flthn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-9m5rm" podUID="15ad29a0-1878-44e9-bc36-819b294b7e61" Sep 10 00:12:25.523797 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.523768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flthn\" (UniqueName: \"kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn\") pod \"authorino-557f876f69-9m5rm\" (UID: \"15ad29a0-1878-44e9-bc36-819b294b7e61\") " pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:25.538921 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.538900 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flthn\" (UniqueName: \"kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn\") pod \"authorino-557f876f69-9m5rm\" (UID: \"15ad29a0-1878-44e9-bc36-819b294b7e61\") " pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:25.549038 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.549014 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-797f55cc8b-9v65w"] Sep 10 00:12:25.552231 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.552211 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.554705 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.554685 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 10 00:12:25.559967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.559947 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-797f55cc8b-9v65w"] Sep 10 00:12:25.725615 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.725579 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7734aaf0-6226-46d7-886b-d13466bafd4c-tls-cert\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.725791 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.725631 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdf27\" (UniqueName: \"kubernetes.io/projected/7734aaf0-6226-46d7-886b-d13466bafd4c-kube-api-access-wdf27\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.826454 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.826373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7734aaf0-6226-46d7-886b-d13466bafd4c-tls-cert\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.826454 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.826416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdf27\" (UniqueName: \"kubernetes.io/projected/7734aaf0-6226-46d7-886b-d13466bafd4c-kube-api-access-wdf27\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.828871 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.828847 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7734aaf0-6226-46d7-886b-d13466bafd4c-tls-cert\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.834846 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.834826 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdf27\" (UniqueName: \"kubernetes.io/projected/7734aaf0-6226-46d7-886b-d13466bafd4c-kube-api-access-wdf27\") pod \"authorino-797f55cc8b-9v65w\" (UID: \"7734aaf0-6226-46d7-886b-d13466bafd4c\") " pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.861794 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.861762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-797f55cc8b-9v65w" Sep 10 00:12:25.949489 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.949457 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:25.957070 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:25.957048 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:26.128509 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.128453 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flthn\" (UniqueName: \"kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn\") pod \"15ad29a0-1878-44e9-bc36-819b294b7e61\" (UID: \"15ad29a0-1878-44e9-bc36-819b294b7e61\") " Sep 10 00:12:26.128509 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.128499 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-797f55cc8b-9v65w"] Sep 10 00:12:26.130454 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.130426 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn" (OuterVolumeSpecName: "kube-api-access-flthn") pod "15ad29a0-1878-44e9-bc36-819b294b7e61" (UID: "15ad29a0-1878-44e9-bc36-819b294b7e61"). InnerVolumeSpecName "kube-api-access-flthn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:12:26.130717 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:12:26.130692 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7734aaf0_6226_46d7_886b_d13466bafd4c.slice/crio-82692b16d8dc66182f67cc8e5968b3c75895bd35ed616f339aea2269a2222d19 WatchSource:0}: Error finding container 82692b16d8dc66182f67cc8e5968b3c75895bd35ed616f339aea2269a2222d19: Status 404 returned error can't find the container with id 82692b16d8dc66182f67cc8e5968b3c75895bd35ed616f339aea2269a2222d19 Sep 10 00:12:26.229456 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.229420 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flthn\" (UniqueName: \"kubernetes.io/projected/15ad29a0-1878-44e9-bc36-819b294b7e61-kube-api-access-flthn\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:12:26.956147 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.956062 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-797f55cc8b-9v65w" event={"ID":"7734aaf0-6226-46d7-886b-d13466bafd4c","Type":"ContainerStarted","Data":"b212746472f247ae733f217dc8084961c103b1bb23729d690e8797941644f09f"} Sep 10 00:12:26.956147 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.956100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-797f55cc8b-9v65w" event={"ID":"7734aaf0-6226-46d7-886b-d13466bafd4c","Type":"ContainerStarted","Data":"82692b16d8dc66182f67cc8e5968b3c75895bd35ed616f339aea2269a2222d19"} Sep 10 00:12:26.956147 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.956117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9m5rm" Sep 10 00:12:26.970625 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.970578 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-797f55cc8b-9v65w" podStartSLOduration=1.420860694 podStartE2EDuration="1.970559962s" totalImagesPullingTime="549.699268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:12:26.968982048 +0000 UTC m=+749.649914780" watchObservedRunningTime="2026-09-10 00:12:26.970559962 +0000 UTC m=+749.651492701" Sep 10 00:12:26.994085 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.994042 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9m5rm"] Sep 10 00:12:26.998808 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.998771 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:12:26.999003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:26.998979 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-m6bkt" podUID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" containerName="authorino" containerID="cri-o://86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c" gracePeriod=30 Sep 10 00:12:27.000854 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.000831 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-9m5rm"] Sep 10 00:12:27.235656 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.235632 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:12:27.340363 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.340329 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmqkw\" (UniqueName: \"kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw\") pod \"9d8fa327-5d97-4c6c-8e49-29513e1fe918\" (UID: \"9d8fa327-5d97-4c6c-8e49-29513e1fe918\") " Sep 10 00:12:27.342319 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.342290 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw" (OuterVolumeSpecName: "kube-api-access-wmqkw") pod "9d8fa327-5d97-4c6c-8e49-29513e1fe918" (UID: "9d8fa327-5d97-4c6c-8e49-29513e1fe918"). InnerVolumeSpecName "kube-api-access-wmqkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:12:27.441984 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.441951 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmqkw\" (UniqueName: \"kubernetes.io/projected/9d8fa327-5d97-4c6c-8e49-29513e1fe918-kube-api-access-wmqkw\") on node \"ip-10-0-128-28.ec2.internal\" DevicePath \"\"" Sep 10 00:12:27.828527 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.828436 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15ad29a0-1878-44e9-bc36-819b294b7e61" path="/var/lib/kubelet/pods/15ad29a0-1878-44e9-bc36-819b294b7e61/volumes" Sep 10 00:12:27.964890 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.964851 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" containerID="86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c" exitCode=0 Sep 10 00:12:27.965341 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.964887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-m6bkt" event={"ID":"9d8fa327-5d97-4c6c-8e49-29513e1fe918","Type":"ContainerDied","Data":"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c"} Sep 10 00:12:27.965341 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.964917 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-m6bkt" Sep 10 00:12:27.965341 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.964929 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-m6bkt" event={"ID":"9d8fa327-5d97-4c6c-8e49-29513e1fe918","Type":"ContainerDied","Data":"a43d26d156a3837a54182ac1e0a6589fdbcc89824fda1bde58484b64e601057f"} Sep 10 00:12:27.965341 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.964949 2582 scope.go:117] "RemoveContainer" containerID="86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c" Sep 10 00:12:27.973851 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.973835 2582 scope.go:117] "RemoveContainer" containerID="86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c" Sep 10 00:12:27.974107 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:12:27.974090 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c\": container with ID starting with 86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c not found: ID does not exist" containerID="86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c" Sep 10 00:12:27.974153 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.974122 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c"} err="failed to get container status \"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c\": rpc error: code = NotFound desc = could not find container \"86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c\": container with ID starting with 86b0ade916e50268a893b419843c544622745b9275e2b21e2700c2c7f22f5f7c not found: ID does not exist" Sep 10 00:12:27.980443 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.980420 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:12:27.983838 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:27.983818 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-m6bkt"] Sep 10 00:12:29.827017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:12:29.826976 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" path="/var/lib/kubelet/pods/9d8fa327-5d97-4c6c-8e49-29513e1fe918/volumes" Sep 10 00:13:08.734616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.734580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jl298"] Sep 10 00:13:08.735008 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.734927 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" containerName="authorino" Sep 10 00:13:08.735008 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.734941 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" containerName="authorino" Sep 10 00:13:08.735079 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.735011 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8fa327-5d97-4c6c-8e49-29513e1fe918" containerName="authorino" Sep 10 00:13:08.739392 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.739376 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.741440 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.741419 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 00:13:08.748887 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.748860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jl298"] Sep 10 00:13:08.881215 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.881160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22fvd\" (UniqueName: \"kubernetes.io/projected/4dc0e6be-2f66-48f8-953f-3c11062eb441-kube-api-access-22fvd\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.881398 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.881240 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc0e6be-2f66-48f8-953f-3c11062eb441-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.982558 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.982501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22fvd\" (UniqueName: \"kubernetes.io/projected/4dc0e6be-2f66-48f8-953f-3c11062eb441-kube-api-access-22fvd\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.982737 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.982596 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc0e6be-2f66-48f8-953f-3c11062eb441-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.983329 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.983302 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc0e6be-2f66-48f8-953f-3c11062eb441-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:08.992760 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:08.992698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22fvd\" (UniqueName: \"kubernetes.io/projected/4dc0e6be-2f66-48f8-953f-3c11062eb441-kube-api-access-22fvd\") pod \"payload-pre-processing-59fbc47bf8-jl298\" (UID: \"4dc0e6be-2f66-48f8-953f-3c11062eb441\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:09.050484 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:09.050451 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:09.325225 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:09.325125 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jl298"] Sep 10 00:13:09.327542 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:13:09.327503 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc0e6be_2f66_48f8_953f_3c11062eb441.slice/crio-2395f6eab60bf02653095f0dd207fcfe270278c1819c4a86fac422f047f09b3d WatchSource:0}: Error finding container 2395f6eab60bf02653095f0dd207fcfe270278c1819c4a86fac422f047f09b3d: Status 404 returned error can't find the container with id 2395f6eab60bf02653095f0dd207fcfe270278c1819c4a86fac422f047f09b3d Sep 10 00:13:10.192587 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:10.192478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" event={"ID":"4dc0e6be-2f66-48f8-953f-3c11062eb441","Type":"ContainerStarted","Data":"2395f6eab60bf02653095f0dd207fcfe270278c1819c4a86fac422f047f09b3d"} Sep 10 00:13:12.203758 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:12.203712 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" event={"ID":"4dc0e6be-2f66-48f8-953f-3c11062eb441","Type":"ContainerStarted","Data":"17ea4468aa253bb559edd9948f85a9f5f9ce668d702abad14c51592c24c9e84b"} Sep 10 00:13:12.204205 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:12.203777 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:12.218069 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:12.218009 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" podStartSLOduration=2.355232155 podStartE2EDuration="4.217989215s" totalImagesPullingTime="1.86275706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:13:12.217596491 +0000 UTC m=+794.898529244" watchObservedRunningTime="2026-09-10 00:13:12.217989215 +0000 UTC m=+794.898921959" Sep 10 00:13:23.211712 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:23.211676 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jl298" Sep 10 00:13:35.748933 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.748894 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv"] Sep 10 00:13:35.753176 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.753159 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.755386 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.755364 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 00:13:35.755569 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.755367 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 10 00:13:35.755569 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.755405 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6p9dv\"" Sep 10 00:13:35.755569 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.755410 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 00:13:35.759594 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.759568 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv"] Sep 10 00:13:35.822185 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.822356 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.822356 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.822356 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822280 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.822356 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822316 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.822356 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.822339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blzdz\" (UniqueName: \"kubernetes.io/projected/543e2ec9-ab3c-4a05-9d6c-daf730b20288-kube-api-access-blzdz\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.922881 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.922847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.922881 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.922884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blzdz\" (UniqueName: \"kubernetes.io/projected/543e2ec9-ab3c-4a05-9d6c-daf730b20288-kube-api-access-blzdz\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.922952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923081 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923066 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923348 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923324 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923350 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.923424 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.923394 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.925282 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.925256 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543e2ec9-ab3c-4a05-9d6c-daf730b20288-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.925387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.925310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543e2ec9-ab3c-4a05-9d6c-daf730b20288-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:35.929450 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:35.929431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blzdz\" (UniqueName: \"kubernetes.io/projected/543e2ec9-ab3c-4a05-9d6c-daf730b20288-kube-api-access-blzdz\") pod \"e2e-distinct-simulated-kserve-6685d86975-gvmgv\" (UID: \"543e2ec9-ab3c-4a05-9d6c-daf730b20288\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:36.064384 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:36.064293 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:36.334290 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:36.334262 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv"] Sep 10 00:13:37.333026 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:37.332990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerStarted","Data":"ae7b7182f1cafd98f496618cbd29fe57deaa02b27d02ea5b19dc29c2e6d51a80"} Sep 10 00:13:38.339160 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:38.339120 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="8a7ba751cb7125a0c2db795691a68704a5ed0129282a94be792f2c62c2480680" exitCode=0 Sep 10 00:13:38.339650 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:38.339202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"8a7ba751cb7125a0c2db795691a68704a5ed0129282a94be792f2c62c2480680"} Sep 10 00:13:38.339650 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:38.339467 2582 scope.go:117] "RemoveContainer" containerID="8a7ba751cb7125a0c2db795691a68704a5ed0129282a94be792f2c62c2480680" Sep 10 00:13:39.345996 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:39.345961 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" exitCode=0 Sep 10 00:13:39.346398 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:39.346046 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7"} Sep 10 00:13:39.346398 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:39.346108 2582 scope.go:117] "RemoveContainer" containerID="8a7ba751cb7125a0c2db795691a68704a5ed0129282a94be792f2c62c2480680" Sep 10 00:13:39.346398 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:39.346375 2582 scope.go:117] "RemoveContainer" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" Sep 10 00:13:39.346615 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:13:39.346593 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:13:40.352699 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:40.352671 2582 scope.go:117] "RemoveContainer" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" Sep 10 00:13:40.353074 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:13:40.352836 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:13:46.065141 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:46.065104 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:46.065549 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:46.065166 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:46.065549 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:46.065182 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:46.065549 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:46.065195 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:13:46.065680 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:46.065578 2582 scope.go:117] "RemoveContainer" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" Sep 10 00:13:46.065769 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:13:46.065750 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:13:56.822289 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:56.822258 2582 scope.go:117] "RemoveContainer" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" Sep 10 00:13:57.447127 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:57.447044 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47" exitCode=0 Sep 10 00:13:57.447266 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:57.447118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47"} Sep 10 00:13:57.447266 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:57.447184 2582 scope.go:117] "RemoveContainer" containerID="57bd7899bb3c95defc23c3acab7b2149c8244944cc659d04f431c132b70515a7" Sep 10 00:13:57.447576 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:13:57.447558 2582 scope.go:117] "RemoveContainer" containerID="a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47" Sep 10 00:13:57.447759 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:13:57.447737 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:06.065388 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:06.065344 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:06.065837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:06.065415 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:06.065837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:06.065431 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:06.065837 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:06.065460 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:06.065940 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:06.065863 2582 scope.go:117] "RemoveContainer" containerID="a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47" Sep 10 00:14:06.066072 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:06.066052 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:13.540961 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.540930 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh"] Sep 10 00:14:13.544692 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.544669 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.546982 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.546960 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 00:14:13.555342 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.555319 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh"] Sep 10 00:14:13.648358 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.648585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhds\" (UniqueName: \"kubernetes.io/projected/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-kube-api-access-mxhds\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.648585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648445 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.648585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.648585 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648544 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.648747 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.648588 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.748983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749017 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhds\" (UniqueName: \"kubernetes.io/projected/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-kube-api-access-mxhds\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749182 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749182 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749080 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749182 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749108 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749182 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749569 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.749660 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.749620 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.751314 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.751288 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.751445 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.751431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.756782 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.756754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhds\" (UniqueName: \"kubernetes.io/projected/87eab194-0a32-47d3-8ac3-fb3a2027cfa9-kube-api-access-mxhds\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh\" (UID: \"87eab194-0a32-47d3-8ac3-fb3a2027cfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:13.856385 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:13.856296 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:14.126294 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:14.126268 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh"] Sep 10 00:14:14.128133 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:14:14.128109 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87eab194_0a32_47d3_8ac3_fb3a2027cfa9.slice/crio-7b143ff57b196e766967962ea4fccc3a95b33939396bc3953026d3c4c0b369a1 WatchSource:0}: Error finding container 7b143ff57b196e766967962ea4fccc3a95b33939396bc3953026d3c4c0b369a1: Status 404 returned error can't find the container with id 7b143ff57b196e766967962ea4fccc3a95b33939396bc3953026d3c4c0b369a1 Sep 10 00:14:14.539730 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:14.539697 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="756802c4606098647c21ad20cdba418841c6a1f4878fc32667c226084f135a78" exitCode=0 Sep 10 00:14:14.539927 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:14.539787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"756802c4606098647c21ad20cdba418841c6a1f4878fc32667c226084f135a78"} Sep 10 00:14:14.539927 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:14.539838 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerStarted","Data":"7b143ff57b196e766967962ea4fccc3a95b33939396bc3953026d3c4c0b369a1"} Sep 10 00:14:14.540079 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:14.540062 2582 scope.go:117] "RemoveContainer" containerID="756802c4606098647c21ad20cdba418841c6a1f4878fc32667c226084f135a78" Sep 10 00:14:15.547312 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:15.547281 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" exitCode=0 Sep 10 00:14:15.547729 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:15.547366 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00"} Sep 10 00:14:15.547729 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:15.547430 2582 scope.go:117] "RemoveContainer" containerID="756802c4606098647c21ad20cdba418841c6a1f4878fc32667c226084f135a78" Sep 10 00:14:15.547817 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:15.547730 2582 scope.go:117] "RemoveContainer" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" Sep 10 00:14:15.547985 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:15.547959 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:14:16.555127 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:16.555103 2582 scope.go:117] "RemoveContainer" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" Sep 10 00:14:16.555509 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:16.555267 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:14:20.449850 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.449819 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79"] Sep 10 00:14:20.453674 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.453651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.455833 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.455807 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 10 00:14:20.461599 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.461576 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79"] Sep 10 00:14:20.509125 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.509260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbf6j\" (UniqueName: \"kubernetes.io/projected/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-kube-api-access-sbf6j\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.509260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.509260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.509370 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.509370 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.509288 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.609784 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.609753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.609967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.609815 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbf6j\" (UniqueName: \"kubernetes.io/projected/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-kube-api-access-sbf6j\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.609967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.609838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.609967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.609856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.609967 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.609881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.610181 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.610051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.610236 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.610176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.610283 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.610274 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.610391 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.610369 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.612147 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.612126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.612301 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.612283 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.616860 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.616838 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbf6j\" (UniqueName: \"kubernetes.io/projected/a8fe3e4e-f11e-4032-8aa1-fbbf529552e7-kube-api-access-sbf6j\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-phf79\" (UID: \"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.763862 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.763827 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:20.822412 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:20.822384 2582 scope.go:117] "RemoveContainer" containerID="a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47" Sep 10 00:14:21.033049 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.033018 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79"] Sep 10 00:14:21.033586 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:14:21.033563 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8fe3e4e_f11e_4032_8aa1_fbbf529552e7.slice/crio-491571c9c2354dd6b019b0e90ac4c54c50db58f755a95d874eed04cb2e8b5153 WatchSource:0}: Error finding container 491571c9c2354dd6b019b0e90ac4c54c50db58f755a95d874eed04cb2e8b5153: Status 404 returned error can't find the container with id 491571c9c2354dd6b019b0e90ac4c54c50db58f755a95d874eed04cb2e8b5153 Sep 10 00:14:21.582457 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.582368 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="bb1c71b03071ed9bbb1bff49c421fdef7bbfcc642b60e6d0aa0e0afc1a49a890" exitCode=0 Sep 10 00:14:21.582892 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.582453 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"bb1c71b03071ed9bbb1bff49c421fdef7bbfcc642b60e6d0aa0e0afc1a49a890"} Sep 10 00:14:21.582892 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.582492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerStarted","Data":"491571c9c2354dd6b019b0e90ac4c54c50db58f755a95d874eed04cb2e8b5153"} Sep 10 00:14:21.582892 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.582732 2582 scope.go:117] "RemoveContainer" containerID="bb1c71b03071ed9bbb1bff49c421fdef7bbfcc642b60e6d0aa0e0afc1a49a890" Sep 10 00:14:21.584702 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.584674 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" exitCode=0 Sep 10 00:14:21.584783 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.584754 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b"} Sep 10 00:14:21.584848 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.584812 2582 scope.go:117] "RemoveContainer" containerID="a6f12ffff55296b4fb30d32e29f07b17d8960504b30140495102636418716c47" Sep 10 00:14:21.585119 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:21.585099 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:14:21.585328 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:21.585310 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:22.593945 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:22.593911 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" exitCode=0 Sep 10 00:14:22.593945 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:22.593948 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580"} Sep 10 00:14:22.594368 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:22.593984 2582 scope.go:117] "RemoveContainer" containerID="bb1c71b03071ed9bbb1bff49c421fdef7bbfcc642b60e6d0aa0e0afc1a49a890" Sep 10 00:14:22.594368 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:22.594245 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:22.594461 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:22.594420 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:23.601816 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.601787 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:23.602254 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:23.602015 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:23.857403 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.857309 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:23.857403 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.857367 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:23.857403 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.857402 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:23.857702 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.857423 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:23.857874 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:23.857857 2582 scope.go:117] "RemoveContainer" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" Sep 10 00:14:23.858057 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:23.858036 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:14:26.065090 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:26.065050 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:26.065483 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:26.065120 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:26.065483 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:26.065137 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:26.065483 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:26.065154 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:14:26.065603 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:26.065490 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:14:26.065704 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:26.065685 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:30.764433 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:30.764400 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:30.764952 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:30.764469 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:30.764952 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:30.764497 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:30.764952 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:30.764509 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:30.764952 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:30.764825 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:30.765082 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:30.764988 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:31.642839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:31.642812 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:31.643016 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:31.642974 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:36.821986 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:36.821948 2582 scope.go:117] "RemoveContainer" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" Sep 10 00:14:36.822416 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:36.822025 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:14:36.822416 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:36.822317 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:37.674447 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:37.674413 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556" exitCode=0 Sep 10 00:14:37.674612 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:37.674484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556"} Sep 10 00:14:37.674612 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:37.674563 2582 scope.go:117] "RemoveContainer" containerID="82e9d681e69927d5ee0805622d2b3605c25d98c0da4b517a455e2c3e317ccf00" Sep 10 00:14:37.674903 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:37.674886 2582 scope.go:117] "RemoveContainer" containerID="141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556" Sep 10 00:14:37.675111 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:37.675088 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:14:43.856476 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:43.856428 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:43.856900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:43.856541 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:43.856900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:43.856569 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:43.856900 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:43.856593 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:14:43.857089 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:43.857073 2582 scope.go:117] "RemoveContainer" containerID="141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556" Sep 10 00:14:43.857288 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:43.857269 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:14:46.822325 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:46.822287 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:47.729593 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:47.729555 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" exitCode=0 Sep 10 00:14:47.729754 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:47.729626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9"} Sep 10 00:14:47.729754 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:47.729675 2582 scope.go:117] "RemoveContainer" containerID="0044f14f59b792d370c202ed3e487a37d9dedf2c9d8a339c4b152652f822f580" Sep 10 00:14:47.730046 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:47.730017 2582 scope.go:117] "RemoveContainer" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" Sep 10 00:14:47.730231 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:47.730209 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:50.764505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:50.764415 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:50.764505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:50.764465 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:50.764505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:50.764480 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:50.764505 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:50.764494 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:14:50.765148 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:50.764973 2582 scope.go:117] "RemoveContainer" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" Sep 10 00:14:50.765208 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:50.765187 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:14:51.822909 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:51.822878 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:14:51.823287 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:51.823073 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:14:57.800565 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:57.800510 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:14:57.801228 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:57.801208 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:14:58.822228 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:58.822193 2582 scope.go:117] "RemoveContainer" containerID="141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556" Sep 10 00:14:59.796207 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:59.796172 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" exitCode=0 Sep 10 00:14:59.796385 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:59.796228 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819"} Sep 10 00:14:59.796385 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:59.796283 2582 scope.go:117] "RemoveContainer" containerID="141ab40a7fc46480294445552e4e852cf8ee09e4f4a89db9ce1962f746eb8556" Sep 10 00:14:59.796650 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:14:59.796633 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:14:59.796852 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:14:59.796829 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:02.823547 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:02.822974 2582 scope.go:117] "RemoveContainer" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" Sep 10 00:15:02.823547 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:02.823312 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:03.856887 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:03.856853 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:03.857293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:03.856914 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:03.857293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:03.856930 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:03.857293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:03.856943 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:03.857293 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:03.857281 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:15:03.857479 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:03.857457 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:05.822485 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:05.822452 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:15:06.837988 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:06.837956 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" exitCode=0 Sep 10 00:15:06.838394 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:06.838016 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678"} Sep 10 00:15:06.838394 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:06.838073 2582 scope.go:117] "RemoveContainer" containerID="d21207acd5171f95bb1c5ade24d5bea6ef11ddc811abccad04cfb1f7da58bf1b" Sep 10 00:15:06.838484 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:06.838421 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:15:06.838634 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:06.838611 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:15:15.822179 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:15.822145 2582 scope.go:117] "RemoveContainer" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" Sep 10 00:15:16.065122 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.065098 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:15:16.065212 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.065156 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:15:16.065212 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.065172 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:15:16.065212 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.065184 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:15:16.065534 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.065503 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:15:16.065696 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:16.065679 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:15:16.822690 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.822652 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:15:16.823098 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:16.822847 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:16.890129 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.890095 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" exitCode=0 Sep 10 00:15:16.890287 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.890143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11"} Sep 10 00:15:16.890287 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.890179 2582 scope.go:117] "RemoveContainer" containerID="0bf9534b5a3872fa35e6665204b38c68fb1329fb1e47f342b9141396b8eff7d9" Sep 10 00:15:16.890560 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:16.890538 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:16.890776 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:16.890754 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:20.764701 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.764668 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:15:20.765188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.764727 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:15:20.765188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.764784 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:15:20.765188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.764825 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:15:20.765188 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.765154 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:20.765391 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:20.765359 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:20.916434 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:20.916402 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:20.916631 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:20.916606 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:26.822711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:26.822673 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:15:26.823098 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:26.822875 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:15:29.822850 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:29.822814 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:15:29.823350 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:29.822992 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:32.822823 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:32.822790 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:32.823259 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:32.823043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:40.822775 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:40.822734 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:15:40.823220 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:40.822924 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:15:43.822590 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:43.822556 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:43.823099 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:43.822612 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:15:43.823099 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:43.822730 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:15:43.823099 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:43.822892 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 00:15:45.043352 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:45.043315 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" exitCode=0 Sep 10 00:15:45.043782 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:45.043385 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07"} Sep 10 00:15:45.043782 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:45.043443 2582 scope.go:117] "RemoveContainer" containerID="4d21c13f41b6bdd6b2fdd22ac73138664566ee0f4935c5a14eef6aa1cec5b819" Sep 10 00:15:45.043880 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:45.043822 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:15:45.044048 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:45.044030 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:52.822434 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:52.822407 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:15:52.822825 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:52.822606 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:15:53.856593 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:53.856555 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:53.857003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:53.856615 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:53.857003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:53.856637 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:53.857003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:53.856658 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:15:53.857131 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:53.857043 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:15:53.857262 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:53.857242 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:15:56.822169 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:56.822135 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:57.112145 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:57.112118 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" exitCode=0 Sep 10 00:15:57.112260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:57.112195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510"} Sep 10 00:15:57.112260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:57.112244 2582 scope.go:117] "RemoveContainer" containerID="e15ec55bf526d194fbafa018f4408bce567eea9b0ef76a34ccbdae173b261f11" Sep 10 00:15:57.112639 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:15:57.112622 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:15:57.112808 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:15:57.112791 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:00.764164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:00.764125 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:16:00.764580 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:00.764179 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:16:00.764580 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:00.764188 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:16:00.764580 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:00.764198 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:16:00.764690 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:00.764585 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:16:00.764764 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:00.764744 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:05.822227 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:05.822193 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:16:05.822707 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:05.822426 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:16:06.822758 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:06.822723 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:16:06.823130 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:06.822888 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:16:13.822241 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:13.822205 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:16:13.822803 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:13.822376 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:17.826435 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:17.826398 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:16:17.826951 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:17.826590 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:16:19.822306 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:19.822268 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:16:19.822701 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:19.822489 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:16:25.822920 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:25.822887 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:16:25.823299 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:25.823071 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:29.823020 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:29.822984 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:16:29.823385 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:29.823248 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:16:33.822462 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:33.822424 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:16:34.300981 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:34.300945 2582 generic.go:356] "Generic (PLEG): container finished" podID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" exitCode=0 Sep 10 00:16:34.301135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:34.301016 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" event={"ID":"543e2ec9-ab3c-4a05-9d6c-daf730b20288","Type":"ContainerDied","Data":"5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58"} Sep 10 00:16:34.301135 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:34.301074 2582 scope.go:117] "RemoveContainer" containerID="8a1fd60176cd9839144dd3e1c46b7d7c061ebf46a044b878a51ee4ac90cfa678" Sep 10 00:16:34.301407 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:34.301386 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:16:34.301655 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:34.301636 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:16:36.065107 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:36.065056 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:16:36.065512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:36.065128 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:16:36.065512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:36.065145 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:16:36.065512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:36.065158 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" Sep 10 00:16:36.065512 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:36.065496 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:16:36.065710 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:36.065690 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:16:38.822339 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:38.822307 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:16:38.822749 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:38.822466 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:43.822402 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:43.822364 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:16:43.822851 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:43.822578 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:16:51.822666 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:51.822631 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:16:51.823140 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:51.822835 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:16:52.822091 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:52.822064 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:16:52.822265 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:52.822221 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:16:57.825764 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:16:57.825732 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:16:57.826204 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:16:57.825978 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:05.822384 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:05.822349 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:17:05.822803 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:05.822571 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:17:07.824412 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:07.824385 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:17:07.824824 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:07.824581 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:17:11.822392 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:11.822354 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:17:12.495836 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:12.495800 2582 generic.go:356] "Generic (PLEG): container finished" podID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" exitCode=0 Sep 10 00:17:12.496006 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:12.495858 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" event={"ID":"87eab194-0a32-47d3-8ac3-fb3a2027cfa9","Type":"ContainerDied","Data":"81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75"} Sep 10 00:17:12.496006 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:12.495929 2582 scope.go:117] "RemoveContainer" containerID="238a8da2d319a1f57cfbdc575247f243da6ef242dae67880946a07796c105c07" Sep 10 00:17:12.496294 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:12.496274 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:17:12.496506 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:12.496475 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:13.856879 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:13.856847 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:17:13.857300 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:13.856896 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:17:13.857300 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:13.856913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:17:13.857300 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:13.856927 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" Sep 10 00:17:13.857300 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:13.857243 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:17:13.857462 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:13.857405 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:19.822469 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:19.822434 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:17:19.822875 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:19.822644 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:17:21.822294 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:21.822257 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:17:22.550473 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:22.550440 2582 generic.go:356] "Generic (PLEG): container finished" podID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" exitCode=0 Sep 10 00:17:22.550649 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:22.550508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" event={"ID":"a8fe3e4e-f11e-4032-8aa1-fbbf529552e7","Type":"ContainerDied","Data":"76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9"} Sep 10 00:17:22.550649 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:22.550582 2582 scope.go:117] "RemoveContainer" containerID="025269956a4fd34eb80427c10e281fda1bb27808b85f72a066432f8ae49d0510" Sep 10 00:17:22.550957 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:22.550940 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:17:22.551126 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:22.551100 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:17:25.828406 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:25.828375 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:17:25.828898 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:25.828588 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:30.764672 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.764628 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:17:30.764672 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.764678 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:17:30.765190 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.764692 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:17:30.765190 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.764702 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" Sep 10 00:17:30.765190 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.765027 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:17:30.765290 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:30.765216 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:17:30.822829 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:30.822800 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:17:30.822996 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:30.822966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:17:39.822415 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:39.822381 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:17:39.822929 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:39.822596 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:43.822260 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:43.822226 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:17:43.822685 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:43.822267 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:17:43.822685 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:43.822410 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:17:43.822685 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:43.822443 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:17:51.822578 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:51.822468 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:17:51.823046 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:51.822729 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:17:55.822232 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:55.822197 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:17:55.822675 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:55.822376 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:17:56.822882 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:17:56.822847 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:17:56.823327 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:17:56.823034 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:18:05.822995 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:05.822959 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:18:05.823381 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:05.823174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:18:06.822182 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:06.822145 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:18:06.822364 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:06.822314 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:18:09.822164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:09.822133 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:18:09.822570 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:09.822308 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:18:17.828151 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:17.828119 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:18:17.828539 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:17.828286 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:18:20.822441 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:20.822406 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:18:20.822960 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:20.822612 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:18:23.906708 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:23.906658 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_87eab194-0a32-47d3-8ac3-fb3a2027cfa9/main/5.log" Sep 10 00:18:24.022000 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:24.021969 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_87eab194-0a32-47d3-8ac3-fb3a2027cfa9/main/5.log" Sep 10 00:18:24.822238 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:24.822204 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:18:24.822421 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:24.822384 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:18:25.095904 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:25.095810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797f55cc8b-9v65w_7734aaf0-6226-46d7-886b-d13466bafd4c/authorino/0.log" Sep 10 00:18:29.822941 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:29.822908 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:18:29.823394 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:29.823084 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:18:31.207363 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:31.207332 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54f95565dd-g5k6r_73d63671-328a-432b-9844-52e214e00c35/copy-manifests/0.log" Sep 10 00:18:31.213757 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:31.213735 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54f95565dd-g5k6r_73d63671-328a-432b-9844-52e214e00c35/manager/0.log" Sep 10 00:18:32.722573 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:32.722540 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797f55cc8b-9v65w_7734aaf0-6226-46d7-886b-d13466bafd4c/authorino/0.log" Sep 10 00:18:32.821986 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:32.821951 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:18:32.822200 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:32.822176 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:18:33.446084 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:33.446047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-j9vn9_02653c5c-3d42-456b-bc77-df7fd7397a00/manager/0.log" Sep 10 00:18:34.055414 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:34.055385 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5976c68f-4lwss_91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c/kube-auth-proxy/0.log" Sep 10 00:18:34.412602 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:34.412510 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jl298_4dc0e6be-2f66-48f8-953f-3c11062eb441/payload-pre-processing/0.log" Sep 10 00:18:34.652190 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:34.652160 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64cc444bc6-mqvdc_b4df5758-cbe1-44c4-9e0f-defca92e159b/router/0.log" Sep 10 00:18:35.124042 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:35.124011 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-gvmgv_543e2ec9-ab3c-4a05-9d6c-daf730b20288/main/5.log" Sep 10 00:18:35.594889 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:35.594854 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_87eab194-0a32-47d3-8ac3-fb3a2027cfa9/main/5.log" Sep 10 00:18:35.704561 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:35.704509 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-phf79_a8fe3e4e-f11e-4032-8aa1-fbbf529552e7/main/5.log" Sep 10 00:18:39.822219 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:39.822186 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:18:39.822627 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:39.822386 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:18:41.822578 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:41.822547 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:18:41.822953 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:41.822692 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:18:44.696711 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:44.696677 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fc4mk_5068c29a-75f0-4876-b3cb-7de73ef69a18/global-pull-secret-syncer/0.log" Sep 10 00:18:44.833693 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:44.833658 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t8h24_25eab51a-ba59-4c67-bf32-89795665a505/konnectivity-agent/0.log" Sep 10 00:18:44.851690 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:44.851660 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-28.ec2.internal_1a426b2798cb1a9b52d39c4792ff821c/haproxy/0.log" Sep 10 00:18:45.822488 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:45.822453 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:18:45.822893 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:45.822674 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:18:49.108241 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:49.108149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797f55cc8b-9v65w_7734aaf0-6226-46d7-886b-d13466bafd4c/authorino/0.log" Sep 10 00:18:49.302713 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:49.302681 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-j9vn9_02653c5c-3d42-456b-bc77-df7fd7397a00/manager/0.log" Sep 10 00:18:51.029772 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.029741 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-xtpll_8157b04a-9036-4a96-a570-eb8d0e22dba5/cluster-monitoring-operator/0.log" Sep 10 00:18:51.126904 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.126877 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-748bbb5d6c-j9qj2_f0c5ddeb-f017-45fa-8b69-5e9e130420e9/metrics-server/0.log" Sep 10 00:18:51.244835 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.244804 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svbxb_66002a5d-bdc8-4612-afbf-f58a1a7e010a/node-exporter/0.log" Sep 10 00:18:51.266001 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.265970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svbxb_66002a5d-bdc8-4612-afbf-f58a1a7e010a/kube-rbac-proxy/0.log" Sep 10 00:18:51.285953 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.285880 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svbxb_66002a5d-bdc8-4612-afbf-f58a1a7e010a/init-textfile/0.log" Sep 10 00:18:51.394373 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.394340 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-mq5qc_7531d018-aa86-4eb8-808a-093dedfac6c7/kube-rbac-proxy-main/0.log" Sep 10 00:18:51.417173 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.417149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-mq5qc_7531d018-aa86-4eb8-808a-093dedfac6c7/kube-rbac-proxy-self/0.log" Sep 10 00:18:51.438902 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.438875 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-mq5qc_7531d018-aa86-4eb8-808a-093dedfac6c7/openshift-state-metrics/0.log" Sep 10 00:18:51.698776 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.698735 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8qp5q_52d2f86a-89d5-4bd1-932a-1037ed420e57/prometheus-operator/0.log" Sep 10 00:18:51.718839 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.718816 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8qp5q_52d2f86a-89d5-4bd1-932a-1037ed420e57/kube-rbac-proxy/0.log" Sep 10 00:18:51.822315 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.822282 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:18:51.822509 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:51.822489 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:18:51.894458 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.894381 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/thanos-query/0.log" Sep 10 00:18:51.921285 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.921259 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/kube-rbac-proxy-web/0.log" Sep 10 00:18:51.942244 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.942200 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/kube-rbac-proxy/0.log" Sep 10 00:18:51.966138 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.966105 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/prom-label-proxy/0.log" Sep 10 00:18:51.986925 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:51.986898 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/kube-rbac-proxy-rules/0.log" Sep 10 00:18:52.006083 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:52.006058 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f469f46-zs2lw_af94e8ae-f947-424f-ada7-939e161f203d/kube-rbac-proxy-metrics/0.log" Sep 10 00:18:52.822804 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:52.822775 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:18:52.823176 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:52.822942 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:18:53.474971 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:53.474944 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-fgc5x_b03b6f71-f814-4d17-bcd7-f73de5046adf/networking-console-plugin/0.log" Sep 10 00:18:54.081561 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:54.081530 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/1.log" Sep 10 00:18:54.087835 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:54.087809 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wvbvd_ef122cd0-dddf-43ac-9deb-bf8c52a04f8e/console-operator/2.log" Sep 10 00:18:54.625464 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:54.625437 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-55c8b4555d-dmdq6_4352c084-a795-4a87-8dd6-a25948631bea/console/0.log" Sep 10 00:18:54.652738 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:54.652708 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-m6248_2c1ae540-1b98-45a5-93ae-d7eab119984c/download-server/0.log" Sep 10 00:18:55.133156 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.133121 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf"] Sep 10 00:18:55.136890 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.136865 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.139183 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.139160 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6tqvv\"/\"default-dockercfg-wqq9k\"" Sep 10 00:18:55.140005 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.139982 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tqvv\"/\"openshift-service-ca.crt\"" Sep 10 00:18:55.140102 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.139986 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tqvv\"/\"kube-root-ca.crt\"" Sep 10 00:18:55.145390 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.145367 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf"] Sep 10 00:18:55.267656 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.267622 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2mz\" (UniqueName: \"kubernetes.io/projected/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-kube-api-access-bd2mz\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.267843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.267671 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-podres\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.267843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.267738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-sys\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.267843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.267769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-proc\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.267843 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.267797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-lib-modules\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368404 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2mz\" (UniqueName: \"kubernetes.io/projected/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-kube-api-access-bd2mz\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368415 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-podres\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-sys\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368458 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-proc\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368481 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-lib-modules\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368588 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-sys\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368616 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368609 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-proc\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368884 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368664 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-lib-modules\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.368884 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.368669 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-podres\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.375003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.374984 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2mz\" (UniqueName: \"kubernetes.io/projected/4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5-kube-api-access-bd2mz\") pod \"perf-node-gather-daemonset-4xdxf\" (UID: \"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5\") " pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.448235 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.448143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:55.718465 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:55.718435 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf"] Sep 10 00:18:55.720229 ip-10-0-128-28 kubenswrapper[2582]: W0910 00:18:55.720205 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4d5f4230_2af0_4b3a_a5fc_d3994fd6d1b5.slice/crio-c3fc84f85b4f8fed7ed0c0e117db379f869f2802fccd113f8bc2c35ddb37926d WatchSource:0}: Error finding container c3fc84f85b4f8fed7ed0c0e117db379f869f2802fccd113f8bc2c35ddb37926d: Status 404 returned error can't find the container with id c3fc84f85b4f8fed7ed0c0e117db379f869f2802fccd113f8bc2c35ddb37926d Sep 10 00:18:56.058263 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.058218 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" event={"ID":"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5","Type":"ContainerStarted","Data":"4a23bd3a80bff17a6b126931f744b6287554e2376c483dbb2ed52e29b96a1485"} Sep 10 00:18:56.058438 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.058279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" event={"ID":"4d5f4230-2af0-4b3a-a5fc-d3994fd6d1b5","Type":"ContainerStarted","Data":"c3fc84f85b4f8fed7ed0c0e117db379f869f2802fccd113f8bc2c35ddb37926d"} Sep 10 00:18:56.058438 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.058307 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:18:56.072942 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.072893 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" podStartSLOduration=1.072879034 podStartE2EDuration="1.072879034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:18:56.071465938 +0000 UTC m=+1138.752398699" watchObservedRunningTime="2026-09-10 00:18:56.072879034 +0000 UTC m=+1138.753811775" Sep 10 00:18:56.489164 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.489131 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c7r5j_d5e4505d-29bf-4961-9e60-0e37f6b40d54/dns/0.log" Sep 10 00:18:56.508987 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.508947 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c7r5j_d5e4505d-29bf-4961-9e60-0e37f6b40d54/kube-rbac-proxy/0.log" Sep 10 00:18:56.639455 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:56.639419 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5g9z9_dd022f5b-cc5a-409f-89f9-6fe1d42e5748/dns-node-resolver/0.log" Sep 10 00:18:57.142379 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:57.142344 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29816640-2r965_41bbd9ae-02c6-43bb-bf1e-8afaaceb980e/image-pruner/0.log" Sep 10 00:18:57.215749 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:57.215721 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p977b_f1749c27-f505-440c-924a-9952ec646773/node-ca/0.log" Sep 10 00:18:58.203387 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:58.203362 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5976c68f-4lwss_91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c/kube-auth-proxy/0.log" Sep 10 00:18:58.273418 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:58.273394 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jl298_4dc0e6be-2f66-48f8-953f-3c11062eb441/payload-pre-processing/0.log" Sep 10 00:18:58.319057 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:58.319031 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64cc444bc6-mqvdc_b4df5758-cbe1-44c4-9e0f-defca92e159b/router/0.log" Sep 10 00:18:58.822221 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:58.822189 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:18:58.822402 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:18:58.822361 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9" Sep 10 00:18:58.981671 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:58.981643 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-smmrm_dc65a25a-30c0-424a-a577-4bc1f09d73ea/serve-healthcheck-canary/0.log" Sep 10 00:18:59.567443 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:59.567418 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zz872_87eede89-4ffe-4c39-9027-6490743cb16d/kube-rbac-proxy/0.log" Sep 10 00:18:59.590681 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:59.590656 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zz872_87eede89-4ffe-4c39-9027-6490743cb16d/exporter/0.log" Sep 10 00:18:59.609678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:18:59.609651 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zz872_87eede89-4ffe-4c39-9027-6490743cb16d/extractor/0.log" Sep 10 00:19:01.864889 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:01.864856 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54f95565dd-g5k6r_73d63671-328a-432b-9844-52e214e00c35/manager/0.log" Sep 10 00:19:01.884242 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:01.884212 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54f95565dd-g5k6r_73d63671-328a-432b-9844-52e214e00c35/copy-manifests/0.log" Sep 10 00:19:02.073152 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:02.073125 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6tqvv/perf-node-gather-daemonset-4xdxf" Sep 10 00:19:03.019680 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:03.019650 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d4c4f85fc-qnqsv_807d1a33-1a1c-47e1-a6a3-69dbe8b946c5/manager/0.log" Sep 10 00:19:03.822713 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:03.822680 2582 scope.go:117] "RemoveContainer" containerID="76193b6623a29117fa3246ab4bac301f7807293336a720f9ae2409bf41ceb2e9" Sep 10 00:19:03.822917 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:19:03.822871 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-phf79_llm(a8fe3e4e-f11e-4032-8aa1-fbbf529552e7)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-phf79" podUID="a8fe3e4e-f11e-4032-8aa1-fbbf529552e7" Sep 10 00:19:04.822876 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:04.822847 2582 scope.go:117] "RemoveContainer" containerID="5fd66156738e94a19ba9006d6683f6a933927db5c53fa2acf2765a7222c4bb58" Sep 10 00:19:04.823242 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:19:04.823043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gvmgv_llm(543e2ec9-ab3c-4a05-9d6c-daf730b20288)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gvmgv" podUID="543e2ec9-ab3c-4a05-9d6c-daf730b20288" Sep 10 00:19:05.049292 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:05.049202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5976c68f-4lwss_91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c/kube-auth-proxy/0.log" Sep 10 00:19:05.102324 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:05.102300 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jl298_4dc0e6be-2f66-48f8-953f-3c11062eb441/payload-pre-processing/0.log" Sep 10 00:19:05.136187 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:05.136157 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64cc444bc6-mqvdc_b4df5758-cbe1-44c4-9e0f-defca92e159b/router/0.log" Sep 10 00:19:06.306808 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:06.306778 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5976c68f-4lwss_91fb8eb6-8bdf-450f-b22a-ae82d26e9c2c/kube-auth-proxy/0.log" Sep 10 00:19:06.358091 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:06.358062 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jl298_4dc0e6be-2f66-48f8-953f-3c11062eb441/payload-pre-processing/0.log" Sep 10 00:19:06.393678 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:06.393592 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64cc444bc6-mqvdc_b4df5758-cbe1-44c4-9e0f-defca92e159b/router/0.log" Sep 10 00:19:09.882361 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.882330 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/kube-multus-additional-cni-plugins/0.log" Sep 10 00:19:09.902231 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.902198 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/egress-router-binary-copy/0.log" Sep 10 00:19:09.923068 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.923044 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/cni-plugins/0.log" Sep 10 00:19:09.941877 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.941848 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/bond-cni-plugin/0.log" Sep 10 00:19:09.960764 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.960735 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/routeoverride-cni/0.log" Sep 10 00:19:09.979003 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.978975 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/whereabouts-cni-bincopy/0.log" Sep 10 00:19:09.997729 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:09.997702 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ldwtj_40e30186-ac33-4218-9719-62255d7f2414/whereabouts-cni/0.log" Sep 10 00:19:10.253459 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:10.253427 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s8dlv_69c0d046-adc2-4166-9ce8-f2fd42ff33e8/kube-multus/0.log" Sep 10 00:19:10.271742 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:10.271699 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bxlwt_2cb4843b-acb3-45bd-98c1-11acccb93f3a/network-metrics-daemon/0.log" Sep 10 00:19:10.293899 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:10.293871 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bxlwt_2cb4843b-acb3-45bd-98c1-11acccb93f3a/kube-rbac-proxy/0.log" Sep 10 00:19:11.581274 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.581242 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/ovn-controller/0.log" Sep 10 00:19:11.597121 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.597095 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/ovn-acl-logging/0.log" Sep 10 00:19:11.621605 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.621576 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/kube-rbac-proxy-node/0.log" Sep 10 00:19:11.643067 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.643045 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 00:19:11.658507 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.658483 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/northd/0.log" Sep 10 00:19:11.676695 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.676668 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/nbdb/0.log" Sep 10 00:19:11.695299 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.695281 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/sbdb/0.log" Sep 10 00:19:11.731720 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.731693 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w8rrn_4dd25809-0f6c-4ff3-86af-c634bbea8253/ovnkube-controller/0.log" Sep 10 00:19:11.821920 ip-10-0-128-28 kubenswrapper[2582]: I0910 00:19:11.821885 2582 scope.go:117] "RemoveContainer" containerID="81f5d9b635de368cb262a156fea7fe998e43f2c475c62b30747c934f2146cf75" Sep 10 00:19:11.822132 ip-10-0-128-28 kubenswrapper[2582]: E0910 00:19:11.822107 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh_llm(87eab194-0a32-47d3-8ac3-fb3a2027cfa9)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-fcjxh" podUID="87eab194-0a32-47d3-8ac3-fb3a2027cfa9"